Data processing apparatus, data processing method, and program

ABSTRACT

A data processing apparatus that processes image data of a moving image includes: an acquiring unit acquiring variations representing degrees of change in the image data, in a unit of picture; a display-type determining unit determining, on the basis of the variations in pictures, display types, which represent display methods in displaying the pictures, in a unit of picture; a display-type acquiring unit acquiring a display type for a picture designated by a picture-designation operating unit operated in designating a picture to be displayed; and a display control unit causing a display to display a picture with a display method represented by a display type for the picture.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present invention contains subject matter related to Japanese PatentApplication JP 2006-040171 filed in the Japanese Patent Office on Feb.17, 2006, the entire contents of which being incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a data processing apparatus, a dataprocessing method, and a program, and, more particularly to, forexample, a data processing apparatus, a data processing method, and aprogram that make it possible to appropriately perform variable speedreproduction including so-called scrub.

2. Description of the Related Art

In recent years, as AV apparatuses such as a recorder and a video camerathat treat image data of moving images, AV apparatuses that can treat HD(High Definition) image data have been realized both for thebroadcasting industry and for consumer use.

In such AV apparatuses, for example, HD image data is encoded by theMPEG (Moving Picture Experts Group) 2 system, the MPEG IMX50 system, andthe like in a unit of long GOP (Group Of Picture) (HD Long GOP). Encodeddata obtained as a result of the encoding is recorded in recording mediasuch as a tape, a hard disk, and an optical disk. Further, in the AVapparatuses, the encoded data recorded in the recording media issubjected to MPEG decoding and (an image corresponding to) image dataobtained as a result of the MPEG decoding is outputted to a display anddisplayed.

The long GOP is, for example, a GOP having fifteen or more pictures asone GOP. The MPEG IMX50 system is an encoding system having a maximumbit rate of 50 Mbps and conforming to MPEG2 4:2:2 P@ML. A format of theMPEG IMX50 system is approved by the SMPTE (Society of Motion Pictureand Television Engineers) and registered in SMPTE365M as a D-10 format.In the MPEG IMX50 system, all pictures are encoded as I pictures.

As the AV apparatuses, other than the recorder and the video camera,there is an editing apparatus that edits a moving image (including anediting program (software) that is executed by a computer to cause thecomputer to function as an editing apparatus).

The editing apparatus that edits a moving image has, in general, afunction of scrub for allowing a user (an editor) to check contents ofthe moving image and search for a so-called edition points (an in pointand an out point).

Scrub means reproduction of a frame or a field that is a picturedesignated by the user. The user designates a picture by, for example,performing operation such as movement of a dial serving as a GUI(Graphical User Interface).

In the editing apparatus having the function of scrub, for example, aframe is designated according to a position of the dial and reproduced.Therefore, when the user performs operation for slowly moving the dial,since a frame designated by (a position of) the dial slowly changes,so-called slow reproduction is performed. When the user performsoperation for quickly moving the dial, since a frame designated by thedial quickly changes, so-called high-speed reproduction is performed.Moreover, when the user does not move the dial, since a frame designatedby the dial does not change, so-called still reproduction is performed(the editing apparatus is in a pause state). Therefore, it can be saidthat the scrub is a kind of variable speed (special) reproduction.

In a technique proposed in JP-A-11-331754, an image having a highresolution is recorded in a random-accessible first recording medium, animage having a low resolution corresponding to the image recorded in thefirst recording medium is recorded in a second recording medium, theimage having a low resolution recorded in the second recording medium isdisplayed in normal reproduction, and the image having a high resolutionrecorded in the first recording medium is displayed at the time of apause.

SUMMARY OF THE INVENTION

When scrub is performed in the editing apparatus with, for example, adisc having encoded data recorded therein as an object, it is necessaryto seek a position on the disc where encoded data of a frame designatedby the dial is recorded, read out the encoded data from the disc, decodethe encoded data as image data, and display (an image corresponding to)the image data on a display.

Therefore, it takes time to display a frame on the display after theframe is designated by the dial. Thus, when encoded data recorded in thedisc is, for example, data having a high bit rate obtained by encodingHD image data in a unit of long GOP, depending on performance of theediting apparatus (the computer that executes an editing program),processing such as decoding is late for display. This causes a large gapbetween a frame presently designated by the dial and a frame displayedon the display. As a result, the user may feel a sense of incongruity(stress).

On the other hand, since so-called nonlinear edition is generallyadopted at present, the function of scrub as means for easily checkingcontents of a moving image and determining edition points is becomingmore and more important. A function of scrub for allowing the user tocheck contents of a moving image more in detail and smoothly isdemanded.

Therefore, it is desirable to appropriately perform variable speedreproduction such as scrub.

According to an embodiment of the invention, there is provided a dataprocessing apparatus that processes image data of a moving image. Thedata processing apparatus includes: display-type determining means fordetermining, on the basis of variations in pictures, display types,which represent display methods in displaying pictures, in a unit ofpicture, the variations representing degrees of change in the imagedata; display-type acquiring means for acquiring a display type for apicture designated by picture-designation operating means operated indesignating a picture to be displayed; and display control means forcausing a display to display a picture with a display method representedby a display type for the picture.

According to the embodiment of the invention, there is provided a dataprocessing method of processing image data of a moving image or aprogram for causing a computer to execute processing of image data of amoving image. The data processing method or the program includes thesteps of: determining, on the basis of variations in pictures, displaytypes, which represent display methods in displaying pictures, in a unitof picture, the variation representing degrees of change in the imagedata; acquiring a display type for a picture designated bypicture-designation operating means operated in designating a picture tobe displayed; and causing a display to display a picture with a displaymethod represented by a display type for the picture.

In the data processing apparatus, the data processing method, or theprogram according to the embodiment, display types, which representdisplay methods in displaying pictures, are determined in a unit ofpicture on the basis of variations in pictures representing degrees ofchange in the image data, in a unit of picture. A display type for apicture designated by picture-designation operating means operated indesignating a picture to be displayed is acquired. A picture isdisplayed with a display method represented by a display type for thepicture.

According to another embodiment of the invention, there is provided adata processing apparatus that processes image data of a moving image.The data processing apparatus includes: display-type acquiring means foracquiring a display type for a picture designated by picture-designationoperating means operated in designating a picture to be displayed on thebasis of display types in a unit of picture that are determined on thebasis of variations in a unit of picture, which represent degrees ofchange in the image data, and represent display methods in displayingpictures, the display types in a unit of picture; and display controlmeans for causing a display to display a picture with a display methodrepresented by a display type for the picture.

According to the embodiment of the invention, there is provided a dataprocessing method of processing image data of a moving image or aprogram for causing a computer to execute processing of image data of amoving image. The data processing method or the program includes thesteps of: acquiring a display type for a picture designated bypicture-designation operating means operated in designating a picture tobe displayed on the basis of display types in a unit of picture that aredetermined on the basis of variations in a unit of picture, whichrepresent degrees of change in the image data, and represent displaymethods in displaying pictures, the display types being; and causing adisplay to display a picture with a display method represented by adisplay type for the picture.

In the data processing apparatus, the data processing method, or theprogram according to the embodiment, a display type for a picturedesignated by picture-designation operating means operated indesignating a picture to be displayed is acquired on the basis ofdisplay types in a unit of picture that are determined on the basis ofvariations in a unit of picture, which represent degrees of change inthe image data, and represent display methods in displaying pictures. Apicture is displayed by a display method represented by a display typefor the picture.

According to still another embodiment of the invention, there isprovided a data processing apparatus that processes image data of amoving image. The data processing apparatus includes: acquiring meansfor acquiring variations representing degrees of change in the imagedata, in a unit of one or more pictures; display-type determining meansfor determining, on the basis of the variations, display types, whichrepresent display methods in displaying a picture, in a unit of pluralpictures; display-type acquiring means for acquiring a display type fora picture designated by picture-designation operating means operated indesignating a picture to be displayed; and display control means forcausing a display to display a picture with a display method representedby a display type for the picture.

In the data processing apparatus according to the embodiment, variationsrepresenting degrees of change in the image data in a unit of one ormore pictures is acquired. Display types, which represent displaymethods in displaying pictures, are determined in a unit of pluralpictures on the basis of the variations. A display type for a picturedesignated by picture-designation operating means operated indesignating a picture to be displayed is acquired. A picture isdisplayed by a display method represented by a display type for thepicture.

According to still another embodiment of the invention, there isprovided a data processing apparatus that processes image data of amoving image. The data processing apparatus includes: acquiring meansfor acquiring display types, which represent display methods indisplaying pictures, determined on the basis of variations in a unit ofone or more pictures representing degrees of change in the image data,in a unit of plural pictures; display-type acquiring means for acquiringa display type for a picture designated by picture-designation operatingmeans operated in designating a picture to be displayed on the basis ofthe display types; and display control means for causing a display todisplay a picture with a display method represented by a display typefor the picture.

In the data processing apparatus according to the embodiment, displaytypes, which represent display methods in displaying pictures,determined on the basis of variations in a unit of one or more picturesrepresenting degrees of change in the image data, in a unit of pluralpictures is acquired. A display type for a picture designated bypicture-designation operating means operated in designating a picture tobe displayed is acquired on the basis of the display type. A picture isdisplayed by a display method represented by a display type for thepicture.

It is possible to record the program in a recording medium.

According to the embodiments of the invention, it is possible toappropriately perform variable speed reproduction such as scrub.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an example of a structure of a dataprocessing system according to an embodiment of the invention;

FIG. 2 is a block diagram showing an example of a configuration ofhardware of a PC 1;

FIG. 3 is a diagram showing an example of display of a viewer 51;

FIG. 4 is a block diagram showing an example of a structure of a camera6;

FIG. 5 is a block diagram showing an example of a structure of anAV-file creating unit 75;

FIG. 6 is a block diagram showing an example of a structure of anFy-file creating unit 76;

FIG. 7 is a diagram for explaining processing of a data reducing unit91;

FIG. 8 is a flowchart for explaining Fy file creation processing;

FIG. 9 is a block diagram showing an example of a structure of avariation calculating unit 92;

FIG. 10 is a diagram for explaining motion information in a unit offrame calculated in a motion-information calculating unit 101;

FIG. 11 is a block diagram showing an example of a structure of themotion-information calculating unit 101;

FIG. 12 is a diagram for explaining processing of the motion-informationcalculating unit 101;

FIG. 13 is a block diagram showing another example of the structure ofthe motion-information calculating unit 101;

FIG. 14 is a diagram for explaining processing of the motion-informationcalculating unit 101;

FIG. 15 is a block diagram showing an example of a structure of afineness-information calculating unit 102;

FIG. 16 is a diagram for explaining processing of thefineness-information calculating unit 102;

FIG. 17 is a block diagram showing another example of the structure ofthe fineness-information calculating unit 102;

FIG. 18 is a diagram for explaining processing of an average calculatingunit 141;

FIG. 19 is a graph of variations for frames calculated by the variationcalculating unit 92;

FIG. 20 is a diagram for explaining an example of a display type;

FIG. 21 is a graph for explaining a display type determined in a unit offrame by a display-type determining unit 93;

FIG. 22 is a block diagram showing an example of a structure of thedisplay-type determining unit 93;

FIG. 23 is a flowchart for explaining display type determinationprocessing;

FIG. 24 is a flowchart for explaining the display type determinationprocessing;

FIG. 25 is a graph of a variation in a unit of frame;

FIG. 26 is a diagram showing an example of an Fy file;

FIG. 27 is a diagram for explaining an example of the display type;

FIG. 28 is a graph of motion information and a display type in a unit offrame;

FIG. 29 is a flowchart for explaining display type determinationprocessing;

FIG. 30 is a flowchart for explaining the display type determinationprocessing;

FIG. 31 is a diagram for explaining an example of a display type;

FIG. 32 is a graph of fineness information and a display type in a unitof frame;

FIG. 33 is a flowchart for explaining display type determinationprocessing;

FIG. 34 is a flowchart for explaining the display type determinationprocessing;

FIG. 35 is a diagram showing an example of an Fy file;

FIG. 36 is a diagram showing an example of an Fy file;

FIG. 37 is a diagram showing an example of an Fy file;

FIG. 38 is a diagram for explaining an example of a display type;

FIG. 39 is a graph of motion information and fineness information and adisplay type;

FIG. 40 is a flowchart for explaining display type determinationprocessing;

FIG. 41 is a diagram showing an example of an Fy file;

FIG. 42 is a block diagram showing an example of a structure of a drive5;

FIG. 43 is a block diagram showing an example of a functional structureof the PC 1 functioning as an editing system;

FIG. 44 is a block diagram showing an example of structures of anFy-file managing unit 214 and a display-type acquiring unit 215;

FIG. 45 is a diagram for explaining a relation between a position of ascrub dial 54 and a frame designated by the position;

FIG. 46 is a diagram showing display methods in scrub of frames of anormal type V2 and a high display rate/low resolution type V3;

FIG. 47 is a diagram for explaining display of a frame, a display typefor which is a still image type V1;

FIG. 48 is a flowchart for explaining processing of scrub;

FIG. 49 is a diagram showing an example of display of an edition window301;

FIG. 50 is a flowchart for explaining GUI display processing;

FIG. 51 is a diagram showing an example of display in an Fy-file displaysection 311;

FIG. 52 is a diagram showing type information displayed again accordingto change of a threshold H or L;

FIG. 53 is a flowchart for explaining GUI display processing;

FIG. 54 is a diagram showing an example of display of the Fy-filedisplay section 311;

FIG. 55 is a diagram showing an example of display of the Fy-filedisplay section 311;

FIG. 56 is a flowchart for explaining GUI display processing;

FIG. 57 is a diagram showing an example of a weight for weighting asection length in a designated section;

FIG. 58 is a diagram showing an example of a weight for weighting asection length in a designated section;

FIG. 59 is a diagram showing designated sections at the time when asection length is not weighted and at the time when a section length isweighted;

FIG. 60 is a flowchart for explaining processing for a section length;

FIG. 61 is a diagram showing an example of display of the Fy-filedisplay section 311;

FIG. 62 is a flowchart for explaining allocation range settingprocessing;

FIG. 63 is a diagram showing an example of display of the viewer 51;

FIG. 64 is a flowchart for explaining display processing;

FIG. 65 is a graph of a variation in each frame forming one GOP; and

FIG. 66 is a diagram for explaining a method of determining a displaytype for each frame forming one GOP.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the invention will be hereinafter explained withreference to the accompanying drawings.

FIG. 1 is a diagram showing an example of a structure of a dataprocessing system (a system means a logical collection of pluralapparatuses regardless of whether apparatuses of respective structuresare provided in an identical housing) according to an embodiment of theinvention.

A PC (Personal Computer) 1 executes various programs to perform variouskinds of processing. At least an OS (Operating System) and an editingprogram serving as software for performing edition of a moving image areinstalled in the PC 1 as programs. The PC 1 executes the editing programunder the control by the OS to function as an editing system (an editingapparatus).

A keyboard 2 and a mouse 3 are operated in inputting a character(string), a command, and other information to the PC 1.

A hard disk (drive) 4 is a so-called external hard disk. In FIG. 1, thehard disk 4 is connected to the PC 1. The hard disk 4 records andreproduces AV (Audio Visual) data and the like under the control by thePC 1. It is possible to use the hard disk 4 for, for example, whenedition of a moving image is performed in the PC 1, ripping of AV dataof the moving image, i.e., saving of the AV data as a file of a formatthat can be processed in the PC 1.

A drive 5 is, for example, an external optical disk drive. An opticaldisk 7 is removably insertable into the drive 5. In FIG. 1, the drive 5is connected to the PC 1 and records AV data and the like in andreproduces AV data and the like from the optical disk 7 under thecontrol of the PC 1.

A camera (a video camera) 6 images a subject and records AV data (imagedata and sound data incidental to the image data) of a moving imageobtained as a result of the imaging in a recording medium such as theoptical disk 7 removably insertable into the camera 6, a not-shownmagnetic tape, or a semiconductor memory or transmits the AV data via awire or wireless transmission medium. The camera 6 may be a so-called HDcamera that outputs HD image data as image data of a moving image or maybe a so-called SD camera that outputs SD (standard Definition) imagedata as image data of a moving image.

The optical disk 7 is inserted in, for example, the camera 6. AV dataand the like obtained by the camera 6 is recorded in the optical disk 7.The optical disk 7 is inserted into the drive 5. AV data and the likerecorded in the optical disk 7 are reproduced. As the optical disk 7, itis possible to adopt, for example, a DVD (Digital Versatile Disc) and aprofessional Disc®.

The professional disc is an optical disk that is capable of performinghigh-density recording by a blue-violet laser. Other than HD or SD imagedata of a moving image that a user originally desires to record, datawith an amount of data reduced by, for example, reducing the number ofpixels compared with that of the HD or SD image data (hereinafterreferred to as proxy data as appropriate) and metadata of the HD or SDimage data are recorded in the professional disc.

As opposed to the proxy data, the HD or SD image data of the movingimage that the user originally desires to record is referred to as mainline data as appropriate.

Besides image data of a moving image, concerning sound data incidentalto the image data (sound data of the moving image), other than sounddata of the moving image that the user originally desires to record,data with an amount of data of the sound data reduced is recorded in theprofessional disc. The main line data means, other than the HD or SDimage data of the moving image that the user originally desires torecord, when necessary, the sound data of the moving image that the useroriginally desires to record or both the image data and the sound data.Similarly, the proxy data means, other than the data with an amount ofdata of the image data serving as the main line data reduced, whennecessary, data with an amount of data of the sound data serving as themain line data reduced or both the data.

FIG. 2 is a diagram showing an example of a structure of hardware of thePC 1 in FIG. 1.

A CPU (Central Processing Unit) 22, a ROM (Read Only Memory) 23, a RAM(Random Access Memory) 24, a hard disk 25, and an input/output interface30 are connected to a bus 21.

The CPU 22 executes programs to perform various kinds of processing. TheROM 23 has stored therein the programs executed by the CPU 22 andnecessary data. The RAM 24 temporarily stores the programs executed bythe CPU 22 and necessary data. The hard disk (drive) 25 is a built-inhard disk. The programs executed by the CPU 22 are installed andnecessary data is recorded in the hard disk 25.

An output unit 26 includes a display 41 such as an LCD (Liquid CrystalDisplay) and a speaker 42. The output unit 26 is connected to theinput/output interface 30. The output unit 26 displays an imagecorresponding to data supplied from the input/output interface 30 andoutputs sound corresponding to the data.

An input unit 27 includes a keyboard 2, a mouse 3, and a not-shownmicrophone. The input unit 27 is connected to the input/output interface30. When the user operates the keyboard 2 or the mouse 3, the input unit27 outputs an operation signal supplied from the keyboard 2 or the mouse3 to the input/output interface 30.

A communication unit 28 is a network interface such as a network card (aLAN (Local Area Network) card) or an ADSL (Asymmetric Digital SubscriberLine) modem. The communication unit 28 performs communication controlbetween the PC 1 and a not-shown network.

A drive 29 is a built-in drive. It is possible to insert a removablerecording medium 31 into and remove the removable recording medium 31from the drive 29. The drive 29 reads data from and writes data in theremovable recording medium 31 inserted therein.

The input/output interface 30 functions as an interface between the bus21 and the output unit 26, the input unit 27, the communication unit 28,the drive 29, and other peripheral devices. In FIG. 2 (FIG. 1), the harddisk 4 and the drive 5 are connected to the input/output interface 30 asthe other peripheral devices.

The removable recording medium 31 is a recording medium removablyinsertable into the drive 29 such as a flexible disc, a CD-ROM (CompactDisc Read Only Memory), an MO (Magneto Optical) disc, a DVD (DigitalVersatile Disc), a magnetic disc, or a semiconductor memory.

In the PC 1 constituted as described above, for example, when the useroperates the input unit 27 via the input/output interface 30 and inputsa command, the CPU 22 executes the programs stored in the ROM (Read OnlyMemory) 23 in accordance with the command. Alternatively, the CPU 22loads programs stored (installed) in the external hard disk 4 or thebuilt-in hard disk 25, programs transferred from a satellite or anetwork, received by the communication unit 28, and installed in thehard disk 4 or 25, or programs read out from the removable recordingmedium 31 inserted in the built-in drive 29 or the optical disk 7inserted in the external drive 5 and installed in the hard disk 4 or 25to the RAM (Random Access Memory) 24 and executes the programs.Consequently, the CPU 22 performs processing conforming to flowchartsdescribed later or processing performed by components of a block diagramdescribe later. For example, the CPU 22 outputs a result of theprocessing from the output unit 26 including the display 41 such as anLCD (Liquid Crystal Display) and the speaker 42, transmits the resultfrom the communication unit 28, or records the result in the hard disk 4or 25 via the input/output interface 30 when necessary.

It is possible to record (install) the programs executed by the CPU 22in the hard disk 25 or the ROM 23 serving as the recording medium builtin the PC 1 or the external hard disk 4 in advance.

Alternatively, it is possible to temporarily or permanently store(record) the programs in the optical disk 7 removably insertable intothe external drive 5 or the removable recording medium 31 removablyinsertable into the built-in drive 29. It is possible to provide theoptical disk 7 and the removable recording medium 31 as so-calledpackage software.

Other than installing the programs in the PC 1 from the removablerecording medium 31 and the like described above, it is possible totransfer the programs from a download site to the PC 1 by radio via anartificial satellite for digital satellite broadcast or by wire vianetworks such as a LAN (Local Area Network) and the Internet. In the PC1, it is possible to receive the programs transferred in this way in thecommunication unit 28 and install the programs in the built-in hard disk25 and the like.

Processing steps that the PC 1 performs by executing the programs do notalways have to be processed in time series according to an orderdescribed as the flowcharts described later. The processing steps alsoinclude processing executed in parallel or individually (e.g., parallelprocessing or processing according to an object).

The programs may be processed only by the PC 1 or may be processed byplural PCs 1 in a distributed manner. Moreover, the programs may betransferred to a remote PC and executed therein.

As described above, at least the OS and the editing program forperforming edition of a moving image are installed in the PC 1. When apower supply is turned on, the PC 1 (the CPU 22) executes the OS andfurther executes the editing program under the control by the OSaccording to operation by the user. Consequently, the PC 1 functions asan editing system.

The PC 1 serving as the editing system causes, in order to show a movingimage (an image) or the like serving as a so-called material, which isan object of edition, to the user, the display 41 to display a windowfor displaying the moving image or the like serving as the material.

FIG. 3 is a diagram showing an example of a structure of a viewer 51serving as the window for displaying the moving image or the likeserving as the material.

The viewer 51 includes a display section 52, a scrub bar 53, a time-codedisplay section 55, and an operation section 56 and has a function ofscrub.

The display section 52 is arranged from the upper part to the center tolower part of the viewer 51 and displays a moving image (an image).

The scrub bar 53 is operated in designating a picture displayed in thedisplay section 52 among pictures forming the moving image. The scrubbar 53 is a GUI (Graphical User Interface) of a bar extending in thehorizontal direction in which a scrub dial 54 thereof is movable. Theuser can move the scrub dial 54 along the scrub bar 53 by, for example,operating the scrub dial 54. The user can designate a picture displayedin the display section 52 according to a position of the scrub dial 54.

The picture means a frame or a field. In the following explanation, aframe is adopted as the picture. However, it is also possible to adopt afield as the picture.

The time-code display section 55 is provided below the scrub bar 53. Atime code of a frame displayed in the display section 52 is displayed inthe time code display section 55.

The operation section 56 is provided below the time-code display section55. The operation section 56 includes a play button operated in startingreproduction of a moving image (display of an image in the displaysection 52), a stop button operated in stopping reproduction and thelike, and a fast forward button operated in fast-forwarding the movingimage.

FIG. 4 is a diagram showing an example of a structure of the camera 6 inFIG. 1.

The camera 6 includes an imaging unit 71, an image processing unit 72, amicrophone 73, a sound processing unit 74, an AV-file creating unit 75,an Fy-file creating unit 76, and a recording unit/communicating unit 77.

The imaging unit 71 images a subject to acquire SD or HD image data andsupplies the image data to the image processing unit 72. The imagingunit 71 includes a photoelectric conversion element such as a CCD(Charge Coupled Device) or a CMOS (Complementary Metal OxideSemiconductor) imager and an optical system such as a lens thatcondenses light on the photoelectric conversion element. The imagingunit 71 photoelectrically converts light made incident thereon andsupplies (digital) image data corresponding to the light to the imageprocessing unit 72.

The image processing unit 72 applies necessary processing such as noiseremoval processing for removing noise to the image data from the imagingunit 71. The image processing unit 72 supplies the image data after theprocessing to the AV-file creating unit 75 and the Fy-file creating unit76 in a unit of frame in time series.

The microphone 73 collects sounds, which is vibration of the air,inputted while imaging is performed by the imaging unit 71, converts thesound into (digital) sound data serving as an electric signal, andsupplies the sound data to the sound processing unit 74.

The sound processing unit 74 applies necessary processing such as noiseremoval processing to the sound data from the microphone 73. The soundprocessing unit 74 supplies the sound data after the processing to theAV-file creating unit 75 in time series.

The AV-file creating unit 75 creates an AV file in which the time-seriesimage data from the image processing unit 72 and the time-series sounddata from the sound processing unit 74 are stored and supplies the AVfile to the recording unit/communicating unit 77.

The Fy-file creating unit 76 calculates a variation representing adegree of change in the image data from the image processing unit 72 ina unit of frame. The Fy-file creating unit 76 determines, on the basisof a variation in a frame, a display type, which represents a displaymethod in displaying the frame in the viewer 51 (FIG. 3), in a unit ofpicture when necessary. The Fy-file creating unit 76 creates a file inwhich one or both of the variation and the display type in a unit offrame are stored (hereinafter referred to as Fy file as appropriate) as(one of) metafiles in which metadata of the time-series image data fromthe image processing unit 72 is stored (metadata files) and supplies thefile to the recording unit/communicating unit 77.

The recording unit/communicating unit 77 records, in the optical disk 7,the AV file supplied from the AV-file creating unit 75 and the Fy-filecorresponding to the AV file supplied from the Fy-file creating unit 76,i.e., the Fy-file created from the image data, which is stored in the AVfile supplied from the AV-file creating unit 75, and supplied from theFy-file creating unit 76 or transmits the AV file and the Fy file via anot-shown transmission medium.

For example, file names indicating that an AV file f_(av) and an Fy filef_(Fy) corresponding to the AV file f_(av) correspond to each other aregiven to the AV file f_(av) and the Fy file f_(Fy). In other words, thefile names of the AV file f_(av) and the Fy file f_(Fy) corresponding tothe AV file f_(av) are, for example, file names in which only extensionsare different (file names that are identical except the extensions).

FIG. 5 is a diagram showing an example of a structure of the AV-filecreating unit 75 in FIG. 4.

In the example of the structure of the AV-file creating unit 75 shown inFIG. 5, the optical disk 7 is a professional disc in which both mainline data and proxy data are recorded.

In FIG. 5, the AV-file creating unit 75 includes a main line encoder 81,a proxy encoder 82, and a file creating unit 83.

The main line encoder 81 includes a main line video encoder 81V and amain line audio encoder 81A.

Image data serving as main line data is supplied to the main line videoencoder 81V from the image processing unit 72 (FIG. 4). The main linevideo encoder 81V encodes the image data serving as main line data fromthe image processing unit 72 in, for example, the MPEG2 (e.g., MPEG IMXsystem) and supplies the image data to the file creating unit 83.

Sound data is supplied to the main line audio encoder 81A from the soundprocessing unit 74 (FIG. 4). The main line audio encoder 81A convertsthe sound data from the sound processing unit 74 into sound data of, forexample, the AES (Audio Engineering Society) 3 system serving as mainline data and supplies the sound data to the file creating unit 83.

The proxy encoder 82 includes a proxy video encoder 82V and a proxyaudio encoder 82A.

Image data serving as main line data is supplied to the proxy videoencoder 82V from the image processing unit 72 (FIG. 4). The proxy videoencoder 82V curtails the number of pixels of the image data serving asmain line data from the image processing unit 72 to calculate, forexample, image data serving as proxy data having a resolution (a spatialresolution) lower than that of the main line data. Moreover, the proxyvideo encoder 82V encodes the image data serving as proxy data in, forexample, the MPEG4 and supplies the image data to the file creating unit83.

Sound data is supplied to the proxy audio encoder 82A from the soundprocessing unit 74 (FIG. 4). The proxy audio encoder 82A converts thesound data from the sound processing unit 74 into, for example, sounddata of the ITU-T (International Telecommunication Union,Telecommunication Standardization Sector) G.711 A-Law system serving asproxy data and supplies the sound data to the file creating unit 83.

The file creating unit 83 multiplexes (the encoded data) of the imagedata serving as main line data from the main line video encoder 81V, thesound data serving as main line data from the main line audio encoder81A, the image data serving as proxy data from the proxy video encoder82V, and the sound data serving as proxy data from the proxy audioencoder 82A in, for example, a reproduction time of about two seconds,respectively. The file creating unit 83 creates an AV file of apredetermined format in which a bit stream obtained by the multiplexingis stored, i.e., an AV file of a predetermined format in which a bitstream obtained by multiplexing (the image data and the sound dataserving as) main line data and (the image data and the sound dataserving as) proxy data in a reproduction time of about two seconds,respectively, is stored. The file creating unit 83 supplies the AV fileto the recording unit/communicating unit 77 (FIG. 4).

FIG. 6 is a diagram showing an example of a structure of the Fy-filecreating unit 76 in FIG. 4.

The Fy-file creating unit 76 includes a data reducing unit 91, avariation calculating unit 92, a display-type determining unit 93, aselecting unit 94, and a file creating unit 95.

An image data in a unit of frame is supplied to the data reducing unit91 from the image processing unit 72 (FIG. 4). The data reducing unit 91curtails the number of pixels of the image data in a unit of frame fromthe image processing unit 72 and supplies image data with the reducednumber of pixels obtained as a result of the curtailment to thevariation calculating unit 92.

The data reducing unit 91 performs curtailment to reduce the horizontaland the vertical numbers of pixels of a frame of the image data from theimage processing unit 72 to, for example, ⅛, respectively.

Therefore, when the frame of the image data supplied from the imageprocessing unit 72 (FIG. 4) to the data reducing unit 91 is formed by,for example, 720×480 pixels arranged vertically and horizontally, thedata reducing unit 91 curtails the pixels of the image data having720×480 pixels from the image processing unit 72, for example, as shownin FIG. 7 to form image data having the horizontal and vertical numbersof pixels reduced to 90×60 pixels, which are ⅛ of 720×480 pixels.

In other words, the data reducing unit 91 divides the image data having720×480 pixels from the image processing unit 72 into, for example,blocks of 8×8 pixels arranged horizontally and vertically and allocatesan average or the like of pixel values of the pixels forming the blocksto the blocks to generate image data having 90×60 pixels.

The number of pixels of the image data from the image processing unit 72is reduced by the data reducing unit 91 for the purpose of reducingburdens of processing after that. When the Fy-file creating unit 76 hassufficient performance and it is unnecessary to reduce burdens of theprocessing, the data reducing unit 91 may supply the image data from theimage processing unit 72 directly to the variation calculating unit 92at the later stage. Alternatively, the data reducing unit 91 does nothave to be provided in the Fy-file creating unit 76.

The variation calculating unit 92 calculates a variation representing adegree of change in the image data supplied from the data reducing unit91 in a unit of frame and supplies the variation to the display-typedetermining unit 93 and the selecting unit 94 in time series. Thevariation calculating unit 92 can calculate one or both of a variationrepresenting a degree of temporal change (hereinafter referred to astemporal variation as appropriate) and a variation representing a degreeof spatial change (hereinafter referred to as spatial variation asappropriate) as the variation in a unit of frame.

The temporal variation represents a degree of change in an image in atime direction. For example, a temporal variation in a frame representsa degree of change in the image between the frame and a frameimmediately preceding the frame, i.e., a degree of motion of the image.Thus, the temporal variation is hereinafter also referred to as motioninformation as appropriate.

The spatial variation represents a degree of change in an image in aspace direction. For example, a spatial variation in a frame representsa degree of change in a pixel value of pixels forming the frame, i.e.,fineness (complexity) of the image. Thus, the spatial variation ishereinafter also referred to as fineness information.

The display-type determining unit 93 determines, on the basis of thevariation in a unit of frame supplied from the variation calculatingunit 92 in time series, a display type, which represents a displaymethod in displaying the frame, in a unit of frame and supplies thedisplay type to the selecting unit 94. Details of the display type willbe described later.

The selecting unit 94 selects one or both of the variation in a unit offrame supplied from the variation calculating unit 92 and the displaytype in a unit of frame supplied from the display-type determining unit93 and supplies one or both of the variation and the display type to thefile creating unit 95.

It is possible to, for example, set in advance or determine according tooperation by the user what kind of selection is performed in theselecting unit 94. When only the variation in a unit of frame suppliedfrom the variation calculating unit 92 is selected, so to speak, fixedlyin the selecting unit 94, the display-type determining unit 93 and theselecting unit 94 do not have to be provided in the Fy-file creatingunit 76.

The file creating unit 95 creates a predetermined format Fy file inwhich one or both of the variation and the display type in a unit offrame supplied from the selecting unit 94 are stored and supplies the Fyfile to the recording unit/communicating unit 77 (FIG. 4).

Processing performed by the Fy-file creating unit 76 in FIG. 6 (Fy-filecreation processing) will be explained with reference to a flowchart inFIG. 8.

First, in step S11, the data reducing unit 91 performs data reductionfor curtailing the number of pixels of image data from the imageprocessing unit 72 (FIG. 4) when necessary. The data reducing unit 91supplies image data as a result of the data reduction to the variationcalculating unit 92. The processing proceeds to step S12.

In step S12, the variation calculating unit 92 calculates, using theimage data from the data reducing unit 91, a variation representing adegree of change in the image data from the image processing unit 72(FIG. 4) in a unit of frame. The variation calculating unit 92 suppliesthe variation to the display-type determining unit 93 and the selectingunit 94 in time series. The processing proceeds to step S13.

In step S13, the display-type determining unit 93 determines, on thebasis of the variation in a unit of frame from the variation calculatingunit 92, a display type, which represents a display method in displayingthe frame, in a unit of frame. The display-type determining unit 93supplies the display type to the selecting unit 94. The processingproceeds to step S14.

In step S14, the selecting unit 94 selects one or both of the variationin a unit of frame supplied from the variation calculating unit 92 andthe display type in a unit of frame supplied from the display-typedetermining unit 93. The selecting unit 94 supplies one or both of thevariation and the display type to the file creating unit 95. Theprocessing proceeds to step S15.

In step S15, the file creating unit 95 creates a predetermined format Fyfile in which one or both of the variation and the display type in aunit of frame supplied from the selecting unit 94 are stored. The filecreating unit 95 supplies the Fy file to the recordingunit/communicating unit 77 (FIG. 4) and ends the processing.

FIG. 9 is a diagram showing an example of a structure of the variationcalculating unit 92 in FIG. 6.

As described above, the variation calculating unit 92 can calculate oneor both of motion information (a temporal variation) representing adegree of temporal change and fineness information (a spatial variation)representing a degree of spatial change as a variation in a unit offrame.

A diagram at the top in FIG. 9 shows an example of a structure of thevariation calculating unit 92 at the time when only motion informationis calculated as a variation in a unit of frame. A second diagram fromthe top in FIG. 9 shows an example of a structure of the variationcalculating unit 92 at the time when only fineness information iscalculated as a variation in a unit of frame. A third diagram from thetop (at the bottom) in FIG. 9 shows an example of a structure of thevariation calculating unit 92 at the time when both the motioninformation and the fineness information are calculated as a variationin a unit of frame.

The variation calculating unit 92 at the top in FIG. 9 has amotion-information calculating unit 101. The second variationcalculating unit 92 from the top in FIG. 9 has a fineness-informationcalculating unit 102. The third variation calculating unit 92 from thetop in FIG. 9 has the motion-information calculating unit 101 and thefineness-information calculating unit 102.

Image data from the data reducing unit 91 (FIG. 6) is supplied to themotion-information calculating unit 101 and the fineness-informationcalculating unit 102.

The motion-information calculating unit 101 calculates motioninformation in a unit of frame using the image data from the datareducing unit 91 and outputs the motion information. Thefineness-information calculating unit 102 calculates finenessinformation in a unit of frame using the image data from the datareducing unit 91 and outputs the fineness information.

The motion information in a unit of frame calculated by themotion-information calculating unit 101 in FIG. 9 will be explained withreference to FIG. 10.

The motion-information calculating unit 101 calculates, for example, asshown in FIG. 10, as motion information of an ith frame from the topamong frames in time series forming a moving image, a variation in theimage from an immediately preceding i−1th frame to the ith frame. Inthis case, motion information of a first frame is not present. However,as the motion information of the first frame, it is possible to adopt,for example, motion information identical with motion information of asecond frame. Alternatively, it is possible to calculate the motioninformation of the first frame assuming that an image of a 0th frame isan image in which all pixel values are 0 or an image identical with theimage of the first frame.

FIG. 11 is a diagram showing an example of a structure of themotion-information calculating unit 101 in FIG. 9.

In FIG. 11, the motion-information calculating unit 101 includes amotion-vector detecting unit 111 and a statistic-amount calculating unit112.

Image data is supplied to the motion-vector detecting unit 111 from thedata reducing unit 91 (FIG. 6). The motion-vector detecting unit 111sequentially sets frames of the image data from the data reducing unit91 as a frame of interest and divides a frame immediately preceding theframe of interest (hereinafter referred to as preceding frame asappropriate) into, for example, blocks of 16×16 pixels (macro-blocks inthe MPEG). Moreover, the motion-vector detecting unit 111 calculates,for each of the macro-blocks of the preceding frame, a motion vectorrepresenting motion from the preceding frame to the frame of interestand supplies the motion vector to the statistic-amount calculating unit112.

The statistic-amount calculating unit 112 calculates a statistic amountof the motion vector calculated for the macro-blocks of the precedingframe from the motion-vector detecting unit 111 and outputs thestatistic amount as motion information of the frame of interest.

Processing of the motion-information calculating unit 101 in FIG. 11will be further explained with reference to FIG. 12.

In the motion-information calculating unit 101, the motion-vectordetecting unit 111 divides the preceding frame into macro-blocks of16×16 pixels and detects, for each of the macro-blocks of the precedingframe, a block of 16×16 pixels of a frame of interest most similar tothe macro-block (hereinafter referred to as similar block). Themotion-vector detecting unit 111 calculates a vector having, forexample, the upper left part of the macro-block as a start point and theupper left part of the similar block as an end point as a motion vectorΔF0(h,v) of the macro-block.

A position of a macro-block hth from the left and vth from the top ofthe preceding frame is represented as F0(h,v) and a position of a blockof 16×16 pixels of a frame of interest in a position moved by the motionvector ΔF0(h,v) of the macro block F0(h,v) from the macro-block F0(h,v),i.e., a similar block is represented as F1(h,v) The motion vectorΔF0(h,v) of the macro-block F0(h,v) is represented by an equationΔF0(h,v)=F1(h,v)−F0(h,v).

The statistic-amount calculating unit 112 calculates, as a statisticamount of the motion vector calculated for the macro-block of thepreceding frame, for example, a sum D0=Σ|ΔF0(h,v)| of sizes |ΔF0(h,v)|of the motion vectors ΔF0(h,v) of all the macro-blocks of the precedingframe and outputs the sum D0 as motion information of the frame ofinterest.

The summation Σ in the sum D0=Σ|ΔF0(h,v)| represents a summationperformed by changing h from 1 to the number of macro-blocks in thehorizontal direction of the preceding frame and changing v from 1 to thenumber of macro-blocks in the vertical direction of the preceding frame.

When the motion vector ΔF0(h,v) of each of the macro-blocks F0(h,v) ofthe preceding frame is large, the motion information D0, which is a sumof motion vectors, is also large. Therefore, when the motion informationD0 of the frame of interest is large, motion of an image of the frame ofinterest is also large (intense).

In the case described above, the sum D0=Σ|ΔF0(h,v)| of the sizes|Δf0(h,v)| of the motion vectors Δf0(h,v) of all the macro-blocks of thepreceding frame is calculated as the statistic amount of the motionvector calculated for the macro-block of the preceding frame. However,as the statistic amount of the motion vector calculated for themacro-block of the preceding frame, besides the sum D0, it is possibleto adopt, for example, distribution of the motion vectors calculated forthe macro-block of the receding frame.

In this case, the statistic-amount calculating unit 112 calculates anaverage Δ_(ave) of the motion vectors ΔF0(h,v) of all the macro-blocksof the preceding frame and calculates distribution σ0 of the motionvectors ΔF0(h,v) of all the macro-blocks F0(h,v) of the preceding frameby calculating, for example, an equation σ0=Σ(Δf0(h,v)−Δ_(ave))².

The summation Σ in the distribution σ0=Σ(Δf0(h,v)−Δ_(ave))² represents asummation performed by changing h from 1 to the number of macro-blocksin the lateral direction of the preceding frame and changing v from 1 tothe number of macro-blocks in the vertical direction of the precedingframe.

Like the sum D0, the distribution σ0 is also large when motion of theimage of the frame of interest is large (intense).

In the case in which encoding of image data is performed in the mainline video encoder 81V in FIG. 5, when the motion vector ΔF0(h,v) iscalculated, the statistic-amount calculating unit 112 (FIG. 11) cancalculate the statistic amount described above using the motion vectorΔF0(h,v) calculated in the main line video encoder 81V. In this case,the motion-vector detecting unit 111 does not have to be provided in themotion-information calculating unit 101 in FIG. 11.

FIG. 13 is a diagram showing another example of the structure of themotion-information calculating unit 101 in FIG. 9.

In FIG. 13, the motion-information calculating unit 101 includes ahistogram creating unit 121, a histogram storing unit 122, and adifference calculating unit 123.

Image data from the data reducing unit 91 (FIG. 6) is supplied to thehistogram creating unit 121. The histogram creating unit 121sequentially sets frames of the image data from the data reducing unit91 as a frame of interest and creates, so to speak, a simple histogramof a pixel value of the frame of interest. The histogram creating unit121 supplies the histogram to the histogram storing unit 122 and thedifference calculating unit 123 as a histogram of the frame of interest.

The histogram storing unit 122 stores the histogram of the frame ofinterest supplied from the histogram creating unit 121. The histogramstoring unit 122 has a storage capacity enough for storing histogramsfor at least two frames. The histogram storing unit 122 stores thehistogram of the frame of interest of this time supplied from thehistogram creating unit 121 and a histogram of a frame of interest ofthe last time, i.e., a preceding frame.

The difference calculating unit 123 calculates a difference absolutevalue sum described later of the histogram of the frame of interestsupplied from the histogram creating unit 121 and the histogram of thepreceding frame stored in the histogram storing unit 122 and outputs thedifference absolute value sum as motion information of the frame ofinterest.

Processing of the motion-information calculating unit 101 in FIG. 13will be further explained with reference to FIG. 14.

When a pixel value of the image data supplied from the data reducingunit 91 (FIG. 6) to the histogram creating unit 121 is represented by,for example, 8 bits that are capable of representing integer values of 0to 255, the histogram creating unit 121 equally divides a range of 0 to255 into, for example, eight small ranges of 0 to 31, 32 to 63, 64 to95, 96 to 127, 128 to 159, 160 to 191, 192 to 223, and 224 to 255. Thehistogram creating unit 121 calculates frequencies of pixels valuesincluded in the respective small ranges to create a simple histogram ofthe frame of interest.

For example, when an i+1th frame is a frame of interest, the differencecalculating unit 123 calculates an absolute value Δ (a shaded portion inFIG. 14) of a difference value between frequencies of an identical smallrange of the i+1th frame, which is the frame of interest, and an ithframe, which is a preceding frame, as shown in FIG. 14. The differencecalculating unit 123 further calculates a sum of absolute values ofdifference values (a difference absolute value sum) ΣΔ of frequenciescalculated for the eight small ranges of the histogram and outputs thesum as motion information of the frame of interest.

When motion of the frame of interest is large (intense) a frequencydistribution of a pixel value of the frame of interest is different froma frequency distribution of a pixel value of the preceding frame.Therefore, when the difference absolute value sum ΣΔ of the frame ofinterest is large, motion of the image of the frame of interest is alsolarge (intense).

FIG. 15 is a diagram showing an example of a structure of thefineness-information calculating unit 102 in FIG. 9.

In FIG. 15, the fineness-information calculating unit 102 includes a DCT(Discrete Cosine Transform) conversion unit 131, a weighting-factorcalculating unit 132, and an integrating unit 133.

Image data from the data reducing unit 91 (FIG. 6) is supplied to theDCT conversion unit 131. The DCT conversion unit 131 sequentially setsframes of the image data from the data reducing unit 91 as a frame ofinterest and divides the frame of interest into, for example, blocks of8×8 pixels. The DCT conversion unit 131 subjects each of the blocks ofthe frame of interest to DCT conversion and supplies 8×8 DCT factorsobtained for each of the blocks to the integrating unit 133.

The weighting-factor calculating unit 132 calculates weights to beapplied to the 8×8 DCT factors of the block and supplies the weights tothe integrating unit 133.

The integrating unit 133 applies the weights supplied from theweighting-factor calculating unit 132 to the 8×8 DCT factors of theblock supplied from the DCT conversion unit 131 and integrates the DCTfactors to calculate integrated values. Moreover, the integrating unit133 calculates a sum of the integrated values calculated for therespective blocks of the frame of interest and outputs the sum asfineness information of the frame of interest.

Processing of the fineness-information calculating unit 102 in FIG. 15will be further explained with reference to FIG. 16.

The left part in FIG. 16 indicates a base image of the DCT conversion.The base image includes 8×8 patterns (frequency components) A patternfurther on the right side and a pattern further on the lower side arepatterns having higher frequency components.

Among the 8×8 DCT factors of the block, a DCT factor F_(i-1,) j−1 ith(i=1, 2, . . . , 8) from the left and jth (j=1, 2, . . . , 8) from thetop represents a degree of inclusion of a frequency component of apattern ith from the left and jth from the top of the base image in theblock.

The right part in FIG. 16 indicates a weight G_(i-1,) j−1 calculated bythe weighting-factor calculating unit 132 in FIG. 15.

The weight G_(i-1,) j−1 is a weight applied to the DCT factor F_(i-1,)j−1. The weighting-factor calculating unit 132 calculates the weightG_(i-1,) j−1 in accordance with, for example, an equation G_(i-1,)j−1=i×j. Therefore, in the weighting-factor calculating unit 132, theweight G_(i-1,) j−1 calculated is larger when the G_(i-1,) j−1 isapplied to the DCT factor F_(i-1,) j−1 of a higher frequency component.

The integrating unit 133 in FIG. 15 multiplies the DCT factor F_(i-1,)j−1 of the block supplied from the DCT conversion unit 131 by the weightG_(i-1,) j−1 supplied from the weighting-factor calculating unit 132 tocalculate a multiplied value G_(i-1,) j−1×F_(i-1,) j−1. The integratingunit 133 integrates multiplied values G_(i-1,) j−1×F_(i-1,) j−1 obtainedfor the 8×8 DCT factors F_(i-1,) j−1 of the block to calculate anintegrated value V=ΣG_(i-1,) j−1×F_(i-1,) j−1. The summation Σ in theintegrated value V=ΣG_(i-1,) j−1×F_(i-1,) j−1 represents a summationperformed by changing i and j from 1 to 8, respectively.

The integrating unit 133 further calculates a sum K of integrated valuesV obtained for all the blocks of the frame of interest and outputs thesum K as fineness information of the frame of interest.

As a higher frequency component is included in the frame of interest,since the fineness information, which is the sum K of the integratevalues V, is larger, an image of the frame of interest is a finer (morecomplicated) image.

FIG. 17 is a diagram showing another example of the structure of thefineness information calculating unit 102 in FIG. 9.

In FIG. 17, the fineness-information calculating unit 102 includes anaverage calculating unit 141, a difference-value calculating unit 142,and an integrating unit 143.

Image data from the data reducing unit 91 (FIG. 6) is supplied to theaverage calculating unit 141 and the difference-value calculating unit142. The average calculating unit 141 sequentially sets frames of theimage data from the data reducing unit 91 as a frame of interest anddivides the frame of interest into blocks of 8×8 pixels, for example, asshown in FIG. 18. Moreover, the average calculating unit 141 calculatesan average of pixel values of the blocks of the frame of interest andsupplies the average to the difference-value calculating unit 142.

When a pixel value of a kth pixel in a raster scan order of the blocksof 8×8 pixels is represented as P_(k), the average calculating unit 141calculates an average P_(ave) in accordance with an equationP_(ave)=1/(8×8)×ΣP_(k). The summation Σ in the averageP_(ave)=1/(8×8)×ΣP_(k) represents a summation performed by changing kfrom 1 to 8×8 (=64).

Like the average calculating unit 141, the difference-value calculatingunit 142 divides the frame of interest into blocks of 8×8 pixels,calculates an absolute value |P_(k)−P_(ave)| of a difference valuebetween each of pixel values P_(k) of the block and the average valueP_(ave) of pixel values of the block supplied from the averagecalculating unit 141, and supplies the absolute value |P_(k)−P_(ave)| tothe integrating unit 143.

The integrating unit 143 integrates absolute values |P_(k)−P_(ave)| ofdifference values calculated for the respective pixels of the blocksupplied from the difference-value calculating unit 142 to calculate anintegrated value Q=Σ|P_(k)−P_(ave)|. The summation Σ in the integratedvalue Q=Σ|P_(k)−P_(ave)| represents a summation performed by changing kfrom 1 to 8×8 (=64).

Moreover, the integrating unit 143 calculates a sum of integrated valuesQ obtained for all the blocks of the frame of interest and outputs thesum as fineness information of the frame of interest.

The sum of integrated values Q calculated for the frame of interest iscalled intra-AC. As a value of the intra-AC is larger, fluctuation inpixel values in the frame of interest is larger. Therefore, as thefineness information, which is the sum of integrated values Q, islarger, an image of the frame of interest is a finer (more complicated)image.

In this embodiment, the variations such as the motion information andthe fineness information are used for (variable speed reproductionincluding) scrub as described later.

Besides, it is possible to use the variations in, for example, detectinga scene change and realizing efficiency of MPEG encoding.

The display-type determining unit 93 in FIG. 6 will be explained withreference to FIGS. 19 to 25.

FIG. 19 is a graph of a variation for each of frames calculated by thevariation calculating unit 92 (FIG. 6). In FIG. 19, the abscissaindicates a frame (a place of a frame from the top) and the ordinateindicates a variation.

When the variation is, for example, motion information, in a frame inwhich motion is intense (large), the variation is also large. In a framein which there is no (little) motion, the variation is small.

The display-type determining unit 93 (FIG. 6) determines, on the basisof a variation in a unit of frame shown in FIG. 19 calculated by thevariation calculating unit 92, a display type, which represents adisplay method in displaying each of frames, out of plural displaytypes.

FIG. 20 is a diagram showing an example of display types determined bythe display-type determining unit 93.

In FIG. 20, as the display types, there are three types, namely, a stillimage type V1 representing display in a still image and a normal type V2and a high display rate/low resolution type V3 in which resolutions indisplaying a frame and display rates in displaying a frame aredifferent.

The resolution in displaying a frame means a spatial resolution of animage displayed on a display device such as the display 41 (FIG. 2). Forexample, concerning image data serving as main line data and proxy data,the main line data has a higher resolution and the proxy data has alower resolution.

The display rate in displaying a frame means a frequency (a period) forupdating display in the display device such as the display 41. Forexample, when display is updated at a display rate identical with aframe rate (e.g., 30 Hz) of a moving image and a frame to be displayedis updated from a frame to the next frame (a frame temporally followinga frame) at a rate identical with the frame rate, the moving image isdisplayed at a single speed.

Moreover, for example, when display is updated at a display rate twiceas high as the frame rate of the moving image and the frame to bedisplayed is updated from a frame to the next frame at a rate twice ashigh as the frame rate, the moving image is displayed at a double speed.

For example, when display is updated at a display rate identical withthe frame rate of the moving image and the frame to be displayed isupdated from a frame to the next frame at a rate twice as high as theframe rate, the moving image is displayed at a double speed as well.

When display is updated at a display rate twice as high as the framerate of the moving image and the frame to be displayed is updated at arate twice as high as the frame rate, all frames forming the movingimage are displayed without missing. However, when display is updated ata display rate identical with the frame rate of the moving image and theframe to be displayed is updated at a rate twice as high as the framerate, the frames forming the moving image are displayed every otherframes, which is a missing frame state.

Therefore, when the frame to be displayed is updated at a rate higherthan the frame rate, it is possible to prevent (reduce) missing frame byincreasing a display rate.

Concerning a frame of the still image type V1, for example, even if aframe to be displayed is changed, as long as an image of the frame to bedisplayed can be regarded as an image identical with (an image of) aframe displayed immediately before the image, (the image) of the framedisplayed immediately before the image is displayed (continued to bedisplayed).

Concerning a frame of the normal type V2, for example, an image isdisplayed at a display rate identical with a frame rate of a movingimage (hereinafter referred to as normal rate as appropriate) and with aresolution identical with that of image data serving as main line data(a resolution higher than that of image data serving as proxy data)(hereinafter referred to as normal resolution as appropriate).

Concerning a frame of the high display rate/low resolution type V3, forexample, an image is displayed at a display rate higher than the normalrate, for example, a display rate twice as high as the normal rate andwith a resolution identical with that of image data serving as proxydata (a resolution lower than that of image data serving as main linedata).

When there are the three types, namely, the still image type V1, thenormal type V2, and the high display rate/low resolution type V3 as thedisplay types as shown in FIG. 20, the variation calculating unit 92calculates motion information representing a degree of temporal changeof a frame as a variation. The display-type determining unit 93compares, for example, a variation (motion information) of each offrames and two predetermined thresholds and determines, on the basis ofa result of the comparison, that a display type for the frame is thestill image type V1, the normal type V2, or the high display rate/lowresolution type V3.

FIG. 21 is a diagram showing a display type determined by thedisplay-type determining unit 93 in a unit of frame on the basis of thevariation in a unit of frame shown in FIG. 19.

The display-type determining unit 93 compares the variation (motioninformation) in a unit of frame with one of a threshold L and athreshold H that are in a relation of L<H. The display-type determiningunit 93 determines the high display rate/low resolution type V3, whichis a kind of a low resolution/high display rate type, as a display typefor frames having motion information equal to or larger than thethreshold H, i.e., a display type for frames in which motion informationindicates that a degree of temporal change is large.

When, for example, missing frame occurs in display of the frame having alarge degree of temporal change, i.e., a frame having large motion inscrub performed in edition, a temporal change in an image tends to beoverlooked. Thus, it is desirable that the frame having large motion isdisplayed at a high display rate.

However, when the display rate is set high, burdens on an apparatus forscrub increases. Moreover, since an amount of data of the frame withlarge motion is large, burdens of processing for the frame is largerthan burdens of processing for a frame not having large motion.

Thus, in this embodiment, the frame having large motion is displayed ata high display rate. However, the high display rate/low resolution typeV3, which is a kind of a low resolution/high display rate type fordisplaying an image having a low resolution is adopted for the frame.The image having a low resolution has a small amount of data comparedwith an image having a high resolution. Thus, it is possible to reduceburdens on the apparatus for scrub while preventing overlooking of atemporal change in an image in edition by displaying the image having alow resolution at a high display rate for the frame having large motion.

On the other hand, the display-type determining unit 93 determines thestill image type V1 representing display in a still image as a displaytype for frames in which motion information is smaller than thethreshold L, i.e., frames in which motion information indicates that adegree of temporal change is small.

For example, frames having a small degree of temporal change, i.e.,frames having no (little) motion continue in scrub performed in edition.In this case, when a frame to be displayed is updated among the frames,even if display on the display device such as the display 41 (FIG. 2) ischanged from (an image of) a frame immediately before the update to (animage of) a frame to be displayed after the update, an image displayedon the display device does not change (rarely changes).

When the frame to be displayed is updated regardless of no change in animage displayed on the display device, if display on the display deviceis changed from a frame displayed immediately before the update to aframe to be displayed after the update in this way, so to speak, excessburdens are imposed on the apparatus for scrub.

In the case in which frames having no (little) motion continue, when aframe to be displayed is updated among the frames, if display on thedisplay device such as the display 41 (FIG. 2) is changed from a framedisplayed immediately before the update to a frame to be displayed afterthe update, an image displayed on the display device may slightlychange. In this case, in a section of the continuous frames having nomotion that are rarely set as an edition point, attention of a user whooperates the apparatus for scrub is drawn, so to speak, uselessly. Thisimposes burdens on the user.

Thus, in this embodiment, the still image type V1 for displaying, evenif a frame to be displayed is changed, (an image of) a frame displayedimmediately before the update as long as an image of the frame to bedisplayed can be regarded as an image identical with that of the framedisplayed immediately before the update is adopted for a frame having nomotion. This makes it possible to prevent excess burdens from beingimposed on the apparatus for scrub and the user who operates theapparatus.

The display-type determining unit 93 determines the normal type V2 as adisplay type for frames other than frames having motion informationequal to or higher than threshold H and frames having motion informationsmaller than the threshold L, i.e., frames having motion informationequal to or larger than the threshold L and smaller than the thresholdH.

The normal type V2 for displaying an image at a normal rate (a displayrate identical with a frame rate of the image) and with a normalresolution is adopted for a frame having some degree of motion but nothaving very large motion.

In (a sequence of) frames in time series forming a moving image, asection in which frames of the still image type V1 continue is referredto as a still image section and a section in which frames of the normaltype V2 continue is referred to as a normal section. A section in whichframes of the high display rate/low resolution type V3 continue isreferred to as a high display rate/low resolution section.

FIG. 22 is a diagram showing an example of a structure of thedisplay-type determining unit 93 in FIG. 6.

In FIG. 22, the display-type determining unit 93 includes a storing unit161, a threshold processing unit 162, a continuity judging unit 163, anda determining unit 164.

A variation in a unit of frame is supplied to the storing unit 161 fromthe variation calculating unit 92 (FIG. 6). The storing unit 161temporarily stores the variation in a unit of frame from the variationcalculating unit 92.

The threshold processing unit 162 compares the variation in a unit offrame stored in the storing unit 161 and (one or both of) the thresholdH and the threshold L and supplies comparison information representing aresult of the comparison to the continuity judging unit 163 and thedetermining unit 164 in a unit of frame.

The continuity judging unit 163 judges, on the basis of the comparisoninformation in a unit of frame from the threshold processing unit 162,for example, whether frames having a variation equal to or larger thanthe threshold H are continuously present by the number of frames N(plural) or more and whether frames having a variation smaller than thethreshold H are continuously present by the number of frames N or more.The continuity judging unit 163 supplies judgment informationrepresenting a result of the judgment to the determining unit 164.

The determining unit 164 determines, on the basis of the comparisoninformation from the threshold processing unit 162 and the judgmentinformation from the continuity judging unit 163, a display type foreach of frames forming a moving image and outputs the display type.

Processing of the display-type determining unit 93 in FIG. 22 will beexplained with reference to flowcharts in FIGS. 23 and 24.

When a variation in a unit of frame is supplied from the variationcalculating unit 92 (FIG. 6), in step S41, the display-type determiningunit 93 causes the storing unit 161 to cache (temporarily store) thevariation in a unit of frame. The processing proceeds to step S42.

For example, motion information is supplied from the variationcalculating unit 92 to the display-type determining unit 93 as avariation. The display-type determining unit 93 determines a displaytype for each of frames out of, for example, the three types, namely,the still image type V1, the normal type V2, and the high displayrate/low resolution type V3 shown in FIG. 20.

In step S42, the threshold processing unit 162 performs thresholdprocessing for comparing the variation (motion information) in a unit offrame stored in the storing unit 161 and (one or both of) the thresholdH and the threshold L. The threshold processing unit 162 suppliescomparison information representing a result of the comparison of thevariation in a unit of frame and the threshold H or the threshold L tothe continuity judging unit 163 in a unit of frame. The processingproceeds to step S43.

In step S43, the continuity judging unit 163 selects, for example, a topframe in a time series order not set as a frame of interest among framesforming a moving image, for which an Fy file is about to be created bythe Fy-file creating unit 76 (FIG. 6), as a frame of interest. Theprocessing proceeds to step S44.

In step S44, the continuity judging unit 163 judges, on the basis of thecomparison information from the threshold processing unit 162, whether avariation in the frame of interest is equal to or larger than thethreshold H and frames having a variation equal to or larger than thethreshold H are continuously present by the number of frames N (plural)or more immediately before, immediately after, or before and after theframe of interest.

When it is judged in step S44 that the variation in the frame ofinterest is equal to or larger than the threshold H and frames having avariation equal to or larger than the threshold H are continuouslypresent by the number of frames N (plural) or more immediately before,immediately after, or before and after the frame of interest, thecontinuity judging unit 163 supplies judgment information representing aresult of the judgment to the determining unit 164. The processingproceeds to step S45.

In step S45, the determining unit 164 determines, on the basis of thejudgment information from the continuity judging unit 163, the highdisplay rate/low resolution type V3. The processing proceeds to step S50as a display type for the frame of interest.

According to steps S44 and S45, when the variation in the frame ofinterest is equal to or larger than the threshold H and, moreover,frames having a variation equal to or larger than the threshold H arecontinuously present by the number of frames N (plural) or moreimmediately before, immediately after, or before and after the frame ofinterest, the high display rate/low resolution type V3 is determined asa display type for the frame of interest. This determination is based onthe following reason.

As described later, when a display type is determined out of threetypes, namely, the still image type V1, the normal type V2, and the highdisplay rate/low resolution type V3, in scrub, an image having a highresolution (an image having a normal resolution) is displayed for framesof the still image type V1 and frames of the normal type V2. In otherwords, an image is displayed by processing image data of main line datahaving a larger amount of data (than that of proxy data). On the otherhand, concerning frames of the high display rate/low resolution type V3,an image having a low resolution is displayed. In other words, an imageis displayed by processing image data of proxy data having a smalleramount of data (than that of main line data).

For example, when the optical disk 7 (FIG. 1) is a professional disc inwhich main line data and proxy data are recorded, scrub is performedusing the main line data or the proxy data recorded in the optical disk7. Then, concerning frames of the still image type V1 or the normal typeV2, it is necessary to read out the main line data from the optical disk7. Concerning frames of the high display rate/low resolution type V3, itis necessary to read out the proxy data from the optical disk 7.

In the optical disk 7, the main line data and the proxy data arerecorded in positions physically apart from each other. Thus, forexample, when frames of the still image type V1 or the normal type V2and frames of the high display rate/low resolution type V3 alternatelyappear, seek is frequently performed at the time of scrub for (datarecorded in) the optical disk 7. Depending on performance of theapparatus for scrub, it may be difficult to smoothly perform display ofa frame designated by operation of the scrub bar 53 (FIG. 3).

Thus, in this embodiment, in order to prevent seek from being frequentlyperformed, display types are determined such that frames of the highdisplay rate/low resolution type V3 are continuously present by thenumber of frames N or more and frames of the still image type V1 or thenormal type V2 are also present by the number of frames N or more.

For example, as described above, in steps S44 and S45, when thevariation in the frame of interest is equal to or larger than thethreshold H and frames having a variation equal to or larger than thethreshold H are continuously present by the number of frames N or moreimmediately before, immediately after, or before and after the frame ofinterest, the high display rate/low resolution type V3 is determined asa display type for the frame of interest.

It is possible to determine the number of frames N taking into account,for example, a longest seek time of the drive 5 and the like for playingthe optical disk 7 and a total number of frames forming a moving imagefor which an Fy file is about to be created by the Fy-file creating unit76 (FIG. 6). The number of frames N may be determined in response tooperation by the user.

When a display type is determined, the number of frames N is a minimumnumber of frames having a variation equal to or larger than thethreshold H that should continue (moreover, as described later, aminimum number of frames having a variation smaller than the threshold Hthat should continue), the number of frames N is hereinafter referred toas a minimum limit number of frames N as appropriate.

Seek is a problem when, in frames in time series forming a moving image,frames of the still image type V1 or the normal type V2 for displayingan image using the main line data and frames of the high displayrate/low resolution type V3 for displaying an image using the proxy dataare frequently switched. When frames of the still image type V1 andframes of the normal type V2, in both of which images are displayedusing the main line data, are switched, seek is not a problem.

On the other hand, when it is judged in step S44 that the variation inthe frame of interest is not equal to or larger than the threshold H or,even if the variation in the frame of interest is equal to or largerthan the threshold H, frames having a variation equal to or larger thanthe threshold H are not continuously present by the minimum limit numberof frames N or more immediately before, immediately after, and beforeand after the frame of interest, the processing proceeds to step S46.The continuity judging unit 163 judges, on the basis of the comparisoninformation from the threshold processing unit 162, whether a variationin the frame of interest is smaller than the threshold H and frameshaving a variation smaller than the threshold H are continuously presentby the minimum limit number of frames N or more immediately before,immediately after, or before and after the frame of interest.

When it is judged in step S46 that the variation in the frame ofinterest is smaller than the threshold H and frames having a variationsmaller than the threshold H are continuously present by the minimumlimit number of frames N or more immediately before, immediately after,or before and after the frame of interest, the continuity judging unit163 supplies judgment information representing a result of the judgmentto the determining unit 164. The processing proceeds to step S47.

The determining unit 164 receives from the continuity judging unit 163the supply of the judgment information representing the result of thejudgment that the variation in the frame of interest is smaller than thethreshold H and frames having a variation smaller than the threshold Hare continuously present by the minimum limit number of frames N or moreimmediately before, immediately after, or before and after the frame ofinterest. Then, in step S47, the determining unit 164 judges, on thebasis of the comparison information from the threshold processing unit162, whether a variation in the frame of interest is equal to or largerthan the threshold L.

When it is judged in step S47 that the variation in the frame ofinterest is equal to or larger than the threshold L, i.e., when thevariation in the frame of interest is equal to or larger than thethreshold L and smaller than the threshold H, the processing proceeds tostep S48. The determining unit 164 determines the normal type V2 as adisplay type for the frame of interest. The processing proceeds to stepS50.

When it is judged in step S47 that the variation in the frame ofinterest is not equal to or larger than the threshold L, i.e., when thevariation in the frame of interest is smaller than the threshold L, theprocessing proceeds to step S49. The determining unit 164 determines thestill image type V1 as a display type for the frame of interest. Theprocessing proceeds to step S50.

According to steps S46 to S49, when the variation in the frame ofinterest is smaller than the threshold H and frames having a variationsmaller than the threshold H are continuously present by the number offrames N or more immediately before, immediately after, or before andafter the frame of interest, if the variation in the frame of interestis equal to or larger than the threshold L, the normal type V2 isdetermined as a display type for the frame of interest. If the variationin the frame of interest is smaller than the threshold L, the stillimage type V1 is determined as a display type for the frame of interest.This is for the purpose of preventing seek from being frequentlyperformed at the time of scrub as described above.

In step S50, the continuity judging unit 163 judges whether there is aframe not selected as a frame of interest yet among frames forming amoving image, for which an Fy file is about to be created by the Fy-filecreating unit 76 (FIG. 6).

When it is judged in step S50 that there is a frame not selected as aframe of interest yet, the processing returns to step S43. The frame notselected as a frame of interest yet is selected as a frame of interestanew. The same processing is repeated.

When it is judged in step S50 that there isn't a frame not selected as aframe of interest yet, the display-type determining unit 93 ends theprocessing.

On the other hand, when it is judged in step S46 that the variation inthe frame of interest is not smaller than the threshold H or, even ifthe variation in the frame of interest is smaller than the threshold H,frames having a variation smaller than the threshold H are notcontinuously present by the minimum limit number of frames N or moreimmediately before, immediately after, and before and after the frame ofinterest, the processing proceeds to step S61 in FIG. 24. The continuityjudging unit 163 judges whether frames having a variation crossing thethreshold H are continuously present by the minimum limit number offrames N or more immediately before, immediately after, or before andafter the frame of interest.

FIG. 25 is a graph of a variation in a unit of frame. In FIG. 25, theabscissa indicates a frame and the ordinate indicates a variation. Onlythe threshold H of the thresholds H and L is shown in the FIG. 25.

When the variation in the frame of interest is equal to or larger thanthe threshold H and frames having a variation equal to or larger thanthe threshold H are continuously present by the number of frames N ormore immediately before, immediately after, or before and after theframe of interest, in step S45 in FIG. 23, the high display rate/lowresolution type V3 is determined as a display type for the frame ofinterest.

When the variation in the frame of interest is smaller than thethreshold H and frames having a variation smaller than the threshold Hare continuously present by the minimum limit number of frames N or moreimmediately before, immediately after, or before and after the frame ofinterest, in step S48 or S49 in FIG. 23, the normal type V2 or the stillimage type V1 is determined as a display type for the frame of interest.

Therefore, the processing in step S61 in FIG. 24 is performed whenframes having a variation equal to or larger than the threshold H arenot continuously present by the number of frames N or more and frameshaving a variation less than the threshold H are not continuouslypresent by the minimum limit number of frames N or more immediatelybefore, immediately after, and before and after the frame of interest.This is a case in which the frame of interest is, for example, a framein a section in which a frame having a variation crossing the thresholdH is present (a section in which frames having a variation equal to orlarger than the threshold H and frames having a variation smaller thanthe threshold H are mixed) as indicated as sections T₁, T₂, T₃, and T₄in FIG. 25 and a section in which the number of continuing frames havinga variation equal to or larger than the threshold H and the number ofcontinuing frames having a variation smaller than the threshold H aresmaller than the minimum limit number of frames N at most (hereinafterreferred to as mixed section as appropriate).

As shown in FIG. 25, the mixed section is typically present between asection in which frames having a variation equal to or larger than thethreshold H are continuously present by the number of frames N or more(hereinafter referred to as large variation section as appropriate) anda section in which frames having a variation smaller than the thresholdH are continuously present by the minimum limit number of frames N ormore (hereinafter referred to small variation section as appropriate),between two large variation sections, or between two small variationsections.

The mixed section can be divided into a section having a section length(the number of frames) equal to or larger than the minimum limit numberof frames N and a section having a section length smaller than theminimum limit number of frames N. Among the mixed sections T₁ to T₄ inFIG. 25, the mixed section T₂ is a section having a section length equalto or larger than the minimum limit number of frames N and the othermixed sections T₁, T₃, and T₄ are sections having a section lengthsmaller than the minimum limit number of frames N.

The mixed section can also be divided into a section between a largevariation section and a small variation section, a section between largevariation sections, and a section between small variation sections.Among the mixed sections T₁ to T₄ in FIG. 25, the mixed sections T₁ andT₂ are sections between large variation sections and the mixed sectionT₃ is a section between a large variation section and a small variationsection. The mixed section T₄ is a section between small variationsections.

Referring back to FIG. 24, as described above, in step S61, thecontinuity judging unit 163 judges whether frames having a variationcrossing the threshold H are continuously present by the minimum limitnumber frames N or more immediately before, immediately after, andbefore and after the frame of interest, i.e., whether a mixed section inwhich the frame of interest is present is a section having a sectionlength equal to or larger than the minimum limit number of frames N.

When it is judged in step S61 that frames having a variation crossingthe threshold H are continuously present by the minimum number of framesN or more immediately before, immediately after, or before and after theframe of interest, i.e., a mixed section in which the frame of interestis present is a section having a section length equal to or larger thanthe minimum limit number of frames N like the mixed section T₂ in FIG.25, the processing proceeds to step S62. The determining unit 164determines, for example, the normal type V2 as a display type for theframe of interest. The processing proceeds to step S50 in FIG. 23.Thereafter, the processing described above is performed.

Concerning frames in a mixed section in which frames having a variationcrossing the threshold H are continuously present by the minimum limitnumber of frames N or more, the normal type V2 for displaying an imageusing (image data serving as) the main line data or the high displayrate/low resolution type V3 for displaying an image using (image dataserving as) the proxy data is determined as all display types for theframes. This makes it possible to prevent seek from being frequentlyperformed.

Thus, in this embodiment, the normal type V2 is determined as alldisplay types for the frames in the mixed section in which frames havinga variation crossing the threshold H are continuously present by theminimum limit number of frames N or more. Other than the normal type V2,it is also possible to determine the high display rate/low resolutiontype V3 as a display type for frames in the mixed section in whichframes having a variation crossing the threshold H are continuouslypresent by the minimum limit number of frames N or more.

When attention is paid only to the prevention of seek from beingfrequently performed, it is also possible to determine the still imagetype V1 for displaying an image using the main line data as a displaytype for frames in the mixed section. However, since the mixed sectionis a section in which frames having a variation crossing the threshold Hare present, it is not preferable to determine the still image type V1,which is a display type for frames having a variation smaller than thethreshold L, as a display type for frames in the mixed section.Therefore, the normal type V2 or the high display rate/low resolutiontype V3 is determined as a display type for frames in the mixed section.

On the other hand, when it is judged in step S61 that frames having avariation crossing the threshold H are not continuously present by theminimum number of frames N or more immediately before, immediatelyafter, and before and after the frame of interest, i.e., a mixed sectionin which a frame of interest is present (hereinafter referred to asmixed section of interest) is a section having a section length smallerthan the minimum limit number of frames N like the mixed sections T₁,T₃, and T₄ in FIG. 25, the processing proceeds to step S63. Thecontinuity judging unit 163 judges whether one of a variation in a frameadjacent to the mixed section of interest among temporally precedingframes on the left side of the mixed section of interest having asection length smaller than the minimum limit number of frames N(hereinafter referred to as left frame as appropriate) and a variationin a frame adjacent to the mixed section of interest among temporallyfollowing frames on the right side of the mixed section of interest(hereinafter referred to as right frame as appropriate) is equal to orlarger than the threshold H and the other is smaller than the thresholdH. In other words, the continuity judging unit 163 judges whether themixed section of interest having a section length smaller than theminimum limit number of frames N is a section between a large variationsection and a small variation section.

When it is judged in step S63 that the mixed section of interest havinga section length smaller than the minimum limit number of frames N is asection between a large variation section and a small variation section,i.e., the mixed section of interest having a section length smaller thanthe minimum limit number of frames N is a mixed section between a largevariation section and a small variation section like the mixed sectionT₃ in FIG. 25, the processing proceeds to step S62. As described above,the determining unit 164 determines the normal type V2 as a display typefor the frame of interest. The processing proceeds to step S50 in FIG.23. Thereafter, the processing described above is performed.

Both the large variation section and the small variation section on bothsides of the mixed section of interest having a section length smallerthan the minimum limit number of frames N are sections having a sectionlength equal to or larger than the minimum limit number of frames N.Moreover, the high display rate/low resolution type V3 for displaying animage using the proxy data is determined as a display type for frames inthe large variation section. The normal type V2 or the still image typeV1 for displaying an image using the main line data is determined as adisplay type for frames in the small variation section.

Concerning the frames in the mixed section of interest having a sectionlength smaller than the minimum limit number of frames N between thelarge variation section and the small variation section, a display typeis identical with the high display rate/low resolution type V3determined as a display type for frames in the large variation sectionor the normal type V2 determined as a display type for frames in thesmall variation section. As a result, frames of the high displayrate/low resolution type V3 or the normal type V2 continue by theminimum limit number of frames N or more. This makes it possible toprevent seek from being frequently performed.

Thus, in this embodiment, the normal type V2 is determined as alldisplay types for the frames in the mixed section of interest in whichframes having a variation crossing the threshold is less than theminimum limit number of frames N and that is between the large variationsection and the small variation section. Other than the normal type V2,it is also possible to determine the high display rate/low resolutiontype V3 as all display types for the frames in the mixed section ofinterest in which frames having a variation crossing the threshold H areless than the minimum limit number of frames N and that is between thelarge variation section and the small variation section.

On the other hand, when it is judged in step S63 that the mixed sectionof interest having a section length smaller than the minimum limitnumber of frames N is not a section between a large variation sectionand a small variation section, the processing proceeds to step S64. Thecontinuity judging unit 163 judges whether both a variation in a frameadjacent to the mixed section of interest among temporally precedingframes on the left side of the mixed section of interest having asection length smaller than the minimum limit number of frames N (a leftframe) and a variation in a frame adjacent to the mixed section ofinterest among temporally following frames on the right side of themixed section of interest (a right frame) are equal to or larger thanthe threshold H. In other words, the continuity judging unit 163 judgeswhether the mixed section of interest having a section length smallerthan the minimum limit number of frames N is a section between largevariation sections.

When it is judged in step S64 that the mixed section of interest havinga section length smaller than the minimum limit number of frames N isnot a section between large variation sections, i.e., when the mixedsection of interest having a section length smaller than the minimumlimit number of frames N is a mixed section between small variationsections like the mixed section T₄ in FIG. 25, the processing proceedsto step S62. As described above, the determining unit 164 determines thenormal type V2 as a display type for the frame of interest. Theprocessing proceeds to step S50 in FIG. 23. Thereafter, the processingdescribed above is performed.

In other words, concerning the frames in the mixed section of interesthaving a section length smaller than the minimum limit number of framesN between two small variation sections, a display type is identical withthe normal type V2 that is determined (could be determined) as a displaytype for frames in the two small variation sections. As a result, framesof the normal type V2 continue by the minimum limit number of frames Nor more. This makes it possible to prevent seek from being frequentlyperformed.

On the other hand, when it is judged in step S64 that the mixed sectionof interest having a section length smaller than the minimum limitnumber of frames N is a section between large variation sections, i.e.,when the mixed section of interest having a section length smaller thanthe minimum limit number of frames N is a mixed section between largevariation sections like the mixed section T₁ in FIG. 25, the processingproceeds to step S65. The determining unit 164 determines the highdisplay rate/low resolution type V3 as a display type for the frame ofinterest. The processing proceeds to step S50 in FIG. 23. Thereafter,the processing described above is performed.

In other words, concerning the frames in the mixed section of interesthaving a section length smaller than the minimum limit number of framesN between two large variation sections, a display type is identical withthe high display rate/low resolution type V3 determined as a displaytype for frames in the two large variation sections. As a result, framesof the high display rate/low resolution type V3 continue by the minimumlimit number of frames N or more. This makes it possible to prevent seekfrom being frequently performed.

FIG. 26 is a diagram showing an example of an Fy file created by thefile creating unit 95 in FIG. 6 when motion information is calculated asa variation by the variation calculating unit 92 in FIG. 6, a displaytype is determined by the display-type determining unit 93 on the basisof the motion information, and both a variation and a display type areselected by the selecting unit 94.

In the Fy file in FIG. 26, from the left to the right, a frame numberindicating a place of a frame from the top, a time code of a framecorresponding to the frame number, motion information serving as avariation in the frame corresponding to the frame number, and a displaytype for the frame corresponding to the frame number are sequentiallyarranged.

In the Fy file in FIG. 26, any one of the three types, namely, the stillimage type V1, the normal type V2, and the high display rate/lowresolution type V3 shown in FIG. 20 is determined as the display type.

In the Fy file in FIG. 26, for determination of the display type, 1 and5 are adopted as the threshold L and the threshold H, respectively. Asthe minimum limit number of frames N, a value calculated by an equationN=F×P from a total number F of frames of a moving image, for which theFy file is created, and a predetermined factor P is adopted. When thetotal number F of frames of a moving image, for which the Fy file iscreated, is, for example, 3000 and the predetermined factor P is, forexample, 0.01, the minimum limit number of frames N is 30 (=3000×0.01).

In FIG. 26, motion information is adopted as a variation and, on thebasis of motion information of a frame, any one of the three types,namely, the still image type V1, the normal type V2, and the highdisplay rate/low resolution type V3 is determined as a display type forthe frame. However, a method of determining a display type for a frameis not limited to this.

FIG. 27 is a diagram showing another example of display types determinedby the display-type determining unit 93 in FIG. 6.

In FIG. 27, as the display types, there are five types, namely, a stillimage type V11 representing display in a still image and a normal typeV12, a high display rate/normal resolution type V13, an ultrahighdisplay rate/low resolution type V14, and an ultra-ultrahigh displayrate/low resolution type V15 in which resolutions in displaying a frameand display rates in displaying a frame are different.

Concerning frames of the still image type V11, like the still image typeV1 in FIG. 20, even if a frame to be displayed is changed, as long as animage of the frame to be displayed can be regarded as an image identicalwith (an image of) a frame displayed immediately before the image, (theimage) of the frame displayed immediately before the image is displayed(continued to be displayed).

Concerning frames of the normal type V12, like the normal type V2 inFIG. 20, an image is displayed at a display rate identical with a framerate of a moving image (a normal rate) and with a resolution identicalwith that of image data serving as main line data (a normal resolution).

Concerning frames of the high display rate/normal resolution type V13,an image is displayed at a display rate higher than the normal rate, forexample, a display rate twice as high as the normal rate and with aresolution identical with that of image data serving as proxy data (thenormal resolution).

Concerning frames of the ultrahigh display rate/low resolution type V14,an image is displayed at a display rate higher than the display rate ofthe high display rate/normal resolution type V13, for example, a displayrate three times as high as the normal rate and with a resolutionidentical with that of image data serving as proxy data (a resolutionlower than that of image data serving as main line data).

Concerning frames of the ultra-ultrahigh display rate/low resolutiontype V15, an image is displayed at a display rate higher than thedisplay rate of the ultrahigh display rate/low resolution type V14, forexample, a display rate four times as high as the normal rate and with aresolution identical with image data serving as proxy data (a resolutionlower than that of image data serving as main line data).

When the normal rate is (about) 30 frames/second identical with that ofthe NTSC (National Television System Committee) system, display rates ofthe normal type V12, the high display rate/normal resolution type V13,the ultrahigh display rate/low resolution type V14, and theultra-ultrahigh display rate/low resolution type V15 are 30frames/second, 60 frames/second, 90 frames/second, and 120frames/second, respectively.

When there are five types, namely, the still image type V11, the normaltype V12, the high display rate/normal resolution type V13, theultrahigh display rate/low resolution type V14, and the ultra-ultrahighdisplay rate/low resolution type V15 as the display types as shown inFIG. 27, the variation calculating unit 92 (FIG. 6) calculates motioninformation representing a degree of temporal change in a frame as avariation. The display-type determining unit 93 (FIG. 6) compares, forexample, motion information of each of frames and four predeterminedthresholds. The display-type determining unit 93 determines, on thebasis of a result of the comparison, the still image type V11, thenormal type V12, the high display rate/normal resolution type V13, theultrahigh display rate/low resolution type V14, or the ultra-ultrahighdisplay rate/low resolution type V15 as a display type for the frame.

FIG. 28 a graph of motion information serving as a variation and adisplay type in a unit of frame determined by the display-typedetermining unit 93 (FIG. 6) in a unit of frame on the basis of thevariation.

In FIG. 28, the abscissa indicates a frame and the ordinate indicates avariation.

The display-type determining unit 93 (FIG. 6) compares a variation(motion information) in a unit of frame with thresholds TH1, TH2, TH3,and TH4, which are in a relation of TH1<TH2<TH3<TH4. The display-typedetermining unit 93 determines the ultra-ultra high display rate/lowresolution type V15, which is a kind of a low resolution/high displayrate type, as a display type for frames having motion information equalto or larger than the threshold TH4, i.e., frames in which motioninformation indicates that a degree of temporal change is extremelylarge. In FIG. 28, the ultra-ultrahigh display rate/low resolution typeV15 is determined as a display type for frames in a section D5 in whichframes having motion information equal to or larger than the thresholdTH4 continue.

The display-type determining unit 93 determines the ultrahigh displayrate/low resolution type V14 that is a kind of a low resolution/highdisplay rate type and in which a display rate is lower than that of theultra-ultrahigh display rate/low resolution type V15 as a display typefor frame shaving motion information equal to or larger than thethreshold TH3 and smaller than the threshold TH4, i.e., frames in whichmotion information indicates that a degree of temporal change is largeclose to extremely large. In FIG. 28, the ultrahigh display rate/lowresolution type V14 is determined as a display type for frames insections D4 ad D6 in which frames having motion information equal to orlarger than the threshold TH3 and smaller than the threshold TH4continue.

The display-type determining unit 93 determines the high displayrate/normal resolution type V13 in which a display rate is lower thanthat of the ultrahigh display rate/low resolution type V14 and aresolution is higher than that of the ultrahigh display rate/lowresolution type V14 as a display type for frames having motioninformation equal to or larger than the threshold TH2 and smaller thanthe threshold TH3, i.e., frames in which motion information indicatesthat a degree of temporal change is not extremely large but is stilllarge. In FIG. 28, the high display rate/normal resolution type V13 isdetermined as a display type for frames in sections D3 and D7 in whichframes having motion information equal to or larger than the thresholdTH2 and smaller than the threshold TH3 continue.

The display-type determining unit 93 determines the normal type V12 inwhich a display rate is lower than that of the high display rate/normalresolution type V13 as a display type for frames having motioninformation equal to or higher than the threshold TH1 and smaller thanthe threshold TH2, i.e., frames in which motion information indicatesthat a degree of temporal change is not very large. In FIG. 28, thenormal type V12 is determined as a display type for frames in sectionsD2 and D8 in which frames having motion information equal to or largerthan the threshold TH1 and smaller than the threshold TH2 continue.

Moreover, the display-type determining unit 93 determines the stillimage type V11 representing display in a still image as a display typefor frames having motion information smaller than the threshold TH1,i.e., frames in which motion information indicates a degree of temporalchange is small. In FIG. 28, the still image type V11 is determined as adisplay type for frames in sections D1 and D9 in which frames havingmotion information smaller than the threshold TH1 continue.

Processing of the display-type determining unit 93 in FIG. 22 in thecase in which the still image type V11, the normal type V12, the highdisplay rate/normal resolution type V13, the ultrahigh display rate/lowresolution type V14, or the ultra-ultrahigh display rate/low resolutiontype V15 shown in FIG. 27 is determined as a display type will beexplained with reference to flowcharts in FIGS. 29 and 30.

When a variation in a unit of frame is supplied from the variationcalculating unit 92 (FIG. 6), in step S81, the display-type determiningunit 93 causes the storing unit 161 (FIG. 22) to cache (temporarilystore) the variation in a unit of frame. The processing proceeds to stepS82.

For example, motion information is supplied from the variationcalculating unit 92 to the display-type determining unit 93 as avariation.

In step S82, the threshold processing unit 162 (FIG. 22) performsthreshold processing for comparing the variation (the motioninformation) in a unit of frame stored in the storing unit 161 and thethreshold TH1, TH2, TH3, or TH4. The threshold processing unit 162supplies comparison information representing a result of the comparisonof the variation in a unit of frame and the threshold TH1, TH2, TH3, orTH4 to the continuity judging unit 163 in a unit of frame. Theprocessing proceeds to step S83.

In step S83, the continuity judging unit 163 (FIG. 22) selects, forexample, a front frame in a time series order not set as a frame ofinterest among frames forming a moving image, for which an Fy file isabout to be created by the Fy-file creating unit 76 (FIG. 6), as a frameof interest. The processing proceeds to step S84.

In step S84, the continuity judging unit 163 judges, on the basis of thecomparison information from the threshold processing unit 162, whether avariation in the frame of interest is equal to or larger than thethreshold TH3 and frames having a variation equal to or larger than thethreshold TH3 are continuously present by the minimum limit number offrames N or more immediately before, immediately after, or before andafter the frame of interest.

When it is judged in step S84 that the variation in the frame ofinterest is equal to or larger than the threshold TH3 and frames havinga variation equal to or larger than the threshold TH3 are continuouslypresent by the minimum limit number of frames N or more immediatelybefore, immediately after, or before and after the frame of interest,the continuity judging unit 163 supplies judgment informationrepresenting a result of the judgment to the determining unit 164 (FIG.22). The processing proceeds to step S85.

The determining unit 164 receives, from the continuity judging unit 163,the supply of the judgment information representing the result of thejudgment that the variation in the frame of interest is equal to orlarger than the threshold TH3 and frames having a variation equal to orlarger than the threshold TH3 are continuously present by the minimumlimit number of frames N or more immediately before, immediately after,or before and after the frame of interest. Then, in step S85, thedetermining unit 164 judges, on the basis of the comparison informationfrom the threshold processing unit 162, whether a variation in the frameof interest is equal to or larger than the threshold TH4.

When it is judged in step S85 that the variation in the frame ofinterest is equal to or larger than the threshold TH4, the processingproceeds to step S86. The determining unit 164 determines theultra-ultrahigh display rate/low resolution type V15 as a display typefor the frame of interest. The processing proceeds to step S93.

When it is judged in step S85 that the variation in the frame ofinterest is not equal to or larger than the threshold TH4, i.e., whenthe variation in the frame of interest is equal to or larger than thethreshold TH3 and smaller than the threshold TH4, the processingproceeds to step S87. The determining unit 164 determines the ultra-highdisplay rate/low resolution type V14 as a display type for the frame ofinterest. The processing proceeds to step S93.

According to steps S84 and S87, when the variation in the frame ofinterest is equal to or larger than the threshold TH3 and, moreover,frames having a variation equal to or larger than the threshold TH3 arecontinuously present by the number of frames N or more immediatelybefore, immediately after, or before and after the frame of interest,the ultra-ultrahigh display rate/low resolution type V15 or theultrahigh display rate/low resolution type V14 is determined as adisplay type for the frame of interest. This determination is based onthe following reason.

When a display type is determined out of the still image type V11, thenormal type V12, the high display rate/normal resolution type V13, theultrahigh display rate/low resolution type V14, and the ultra-ultrahighdisplay rate/low resolution type V15 shown in FIG. 27, in scrub, animage having a high resolution (an image having a normal resolution) isdisplayed for frames of the still image type V11, frames of the normaltype V12, and frames of the high display rate/normal resolution typeV13. In other words, an image is displayed by processing image data ofmain line data having a larger amount of data (than that of proxy data).

On the other hand, concerning frames of the ultrahigh display rate/lowresolution type V14 and frames of the ultra-ultrahigh display rate/lowresolution type V15, an image having a low resolution is displayed. Inother words, an image is displayed by processing image data of proxydata having a smaller amount of data (than that of main line data).

For example, when the optical disk 7 (FIG. 1) is a professional disc inwhich main line data and proxy data are recorded, scrub is performedusing the main line data or the proxy data recorded in the optical disk7. Then, as in the case explained with reference to FIG. 23, in framesin time series forming a moving image, when frames of the ultrahighdisplay rate/low resolution type V14 or the ultra-ultrahigh displayrate/low resolution type V15 for displaying an image using the proxydata and frames of the still image type V11, the normal type V12, or thehigh display rate/normal resolution type V13 for displaying an imageusing the main line data are frequently switched, seek is frequentlyperformed. Thus, it may be difficult to smoothly perform display of aframe designated by operation of the scrub bar 53 (FIG. 3).

In order to prevent seek from being frequently performed, it isnecessary to prevent frames of the ultrahigh display rate/low resolutiontype V14 or the ultra-ultrahigh display rate/low resolution type V15 fordisplaying an image using the proxy data and frames of the still imagetype V11, the normal type V12, or the high display rate/normalresolution type V13 for displaying an image using the main line datafrom being frequently switched

On the other hand, in both frames of the ultrahigh display rate/lowresolution type V14 and frames of the ultra-ultrahigh display rate/lowresolution type V15, an image is displayed using the proxy data. Thus,seek is not a problem concerning switching of the frames of theultrahigh display rate/low resolution type V14 and the frames of theultra-ultrahigh display rate/low resolution type V15.

Moreover, in all frames of the still image type V11, frames of thenormal type V12, and frames of the high display rate/normal resolutiontype V13, an image is displayed using the main line data. Thus, seek isnot a problem either concerning switching of frames of arbitrary onedisplay type and frames of another display type among the frames of thestill image type V11, the frames of the normal type V12, and the framesof the high display rate/normal resolution type V13.

Thus, in FIGS. 29 and 30, switching of frames of a display type fordisplaying an image using the proxy data (the ultrahigh display rate/lowresolution type V14 or the ultra-ultrahigh display rate/low resolutiontype V15) and frames of a display type for displaying an image using themain line data (the still image type V11, the normal type V12, or thehigh display rate/normal resolution type V13) is prevented from beingcaused in a section of frames continuing by at least the minimum limitnumber of frames N. For this purpose, in steps S84 to S87, when avariation in the frame of interest is equal to or larger than thethreshold TH3 and frames having a variation equal to or larger than thethreshold TH3 continue by the minimum limit number of frames N or moreimmediately before, immediately after, or before and after the frame ofinterest, the ultrahigh display rate/low resolution type V14 or theultra-ultrahigh display rate/low resolution type V15 is determined as adisplay type for the frame of interest.

Further, switching of frames of a display type for displaying an imageusing the proxy data and frames of a display type for displaying animage using the main line data is prevented from being caused in asection of frames continuing by at least the minimum limit number offrames N. For this purpose, in steps S88 to S92 described later, when avariation in the frame of interest is smaller than the threshold TH3 andframes having a variation smaller than the threshold TH3 continues bythe minimum limit number of frames N or more immediately before,immediately after, or before and after the frame of interest, the stillimage type V11, the normal type V12, or the high display rate/normalresolution type V13 is determined as a display type for the frame ofinterest.

When it is judged in step S84 that the variation in the frame ofinterest is not equal to or larger than the threshold TH3 or, even ifthe variation in the frame of interest is equal to or larger than thethreshold TH3, frames having a variation equal to or larger than thethreshold TH3 are not continuously present by the minimum limit numberof frames N or more immediately before, immediately after, and beforeand after the frame of interest, the processing proceeds to step S88.The continuity judging unit 163 judges, on the basis of the comparisoninformation from the threshold processing unit 162, whether a variationin the frame of interest is smaller than the threshold TH3 and frameshaving a variation smaller than the threshold TH3 are continuouslypresent by the minimum limit number of frames N or more immediatelybefore, immediately after, or before and after the frame of interest.

When it is judged in step S88 that the variation in the frame ofinterest is smaller than the threshold TH3 and frames having a variationsmaller than the threshold TH3 are continuously present by the minimumlimit number of frames N or more immediately before, immediately after,or before and after the frame of interest, the continuity judging unit163 supplies judgment information representing a result of the judgmentto the determining unit 164. The processing proceeds to step S89.

The determining unit 164 receives, from the continuity judging unit 163,supply of the judgment information representing the result of thejudgment that the variation in the frame of interest is smaller than thethreshold TH3 and frames having a variation smaller than the thresholdTH3 are continuously present by the minimum limit number of frames N ormore immediately before, immediately after, or before and after theframe of interest. Then, in step S89, the determining unit 164 judges,on the basis of the comparison information from the threshold processingunit 162, whether a variation in the frame of interest is smaller thanthe threshold TH1, equal to or larger than the threshold TH1 and smallerthan the threshold TH2, or equal to or larger than the threshold TH2 andsmaller than the threshold TH3.

When it is judged in step S89 that the variation in the frame ofinterest is smaller than the threshold TH1, the processing proceeds tostep S90. The determining unit 164 determines the still image type V11as a display type for the frame of interest. The processing proceeds tostep S93.

When it is judged in step S89 that the variation in the frame ofinterest is equal to or larger than the threshold TH1 and smaller thanthe threshold TH2, the processing proceeds to step S91. The determiningunit 164 determines the normal type V12 as a display type for the frameof interest. The processing proceeds to step S93.

When it is judged in step S89 that the variation in the frame ofinterest is equal to or larger than the threshold TH2 and smaller thanthe threshold TH3, the processing proceeds to step S92. The determiningunit 164 determines the high display rate/normal resolution type V13 asa display type for the frame of interest. The processing proceeds tostep S93.

According to steps S88 to S92, as described above, when a variation inthe frame of interest is smaller than the threshold TH3 and frameshaving a variation smaller than the threshold TH3 are continuouslypresent by the number of frames N or ore immediately before, immediatelyafter, or before and after the frame of interest, the still image typeV11, the normal type V12, or the high display rate/normal resolutiontype V13, for displaying an image using the main line data, isdetermined as a display type for the frame of interest. As describedabove, this is for the purpose of preventing seek from being frequentlyperformed.

In step S93, the continuity judging unit 163 judges whether there is aframe not selected as a frame of interest yet among frames forming amoving image for which an Fy file is about to be created by the Fy-filecreating unit 76 (FIG. 6).

When it is judged in step S93 that there is a frame not selected as aframe of interest yet, the processing returns to step S83. As describedabove, a frame not selected as a frame of interest yet is selected as aframe of interest anew. Thereafter, the same processing is repeated.

When it is judged in step S93 that there isn't a frame not selected as aframe of interest yet, the display-type determining unit 93 ends theprocessing.

On the other hand, when it is judged in step S88 that the variation inthe frame of interest is not smaller than the threshold TH3 or, even ifthe variation in the frame of interest is smaller than the thresholdTH3, frames having a variation smaller than the threshold TH3 are notcontinuously present by the minimum limit number of frames N or moreimmediately before, immediately after, and before and after the frame ofinterest, the processing proceeds to step S101 in FIG. 30. Thecontinuity judging unit 163 judges whether frames having a variationcrossing the threshold TH3 are continuously present by the minimum limitnumber of frames N or more immediately before, immediately after, orbefore and after the frame of interest.

When a variation in the frame of interest is equal to or larger than thethreshold TH3 and frames having a variation equal to or larger than thethreshold TH3 are continuously present by the number of frames N or moreimmediately before, immediately after, or before and after the frame ofinterest, in step S86 or S87 in FIG. 29, the ultrahigh display rate/lowresolution type V14 or the ultra-ultrahigh display rate/low resolutiontype V15 is determined as a display type for the frame of interest.

When a variation in the frame of interest is smaller than the thresholdTH3 and frames having a variation smaller than the threshold TH3 arecontinuously present by the minimum limit number of frames N or moreimmediately before, immediately after, or before and after the frame ofinterest, in steps S90 to S92 in FIG. 29, the still image type V11, thenormal type V12, or the high display rate/normal resolution type V13 isdetermined as a display type for the frame of interest.

Therefore, the processing in step S101 in FIG. 30 is performed whenframes having a variation equal to or larger than the threshold TH3 arenot continuously present by the number of frames N or more and frameshaving a variation smaller than the threshold TH3 are not continuouslypresent by the minimum limit number of frames N or more immediatelybefore, immediately after, and before and after the frame of interest.This is a case in which the frame of interest is a frame in a section inwhich frames having a variation crossing the threshold TH3 are present(a section in which frames having a variation equal to or larger thanthe threshold TH3 and frames having a variation smaller than thethreshold TH3 are mixed) and a section in which the number of continuingframes having a variation equal to or larger than the threshold TH3 andthe number of continuing frames having a variation smaller than thethreshold TH3 are smaller than the minimum limit number of frames N atmost (hereinafter also referred to as mixed section as appropriate).

As in the case explained with reference to FIG. 25, the mixed section istypically present between a section in which frames having a variationequal to or larger than the threshold TH3 are continuously present bythe number of frames N or more (hereinafter also referred to as largevariation section as appropriate) and a section in which frames having avariation smaller than the threshold TH3 are continuously present by theminimum limit number of frames N or more (hereinafter also referred tosmall variation section as appropriate), between two large variationsections, or between two small variation sections.

Consequently, the mixed section can be divided into a section having asection length (the number of frames) equal to or larger than theminimum limit number of frames N and a section having a section lengthsmaller than the minimum limit number of frames N.

The mixed section can also be divided into a section between a largevariation section and a small variation section, a section between largevariation sections, and a section between small variation sections.

In step S101, the continuity judging unit 163 judges whether frameshaving a variation crossing the threshold TH3 are continuously presentby the minimum limit number of frames N or more immediately before,immediately after, or before and after the frame of interest, i.e., amixed section in which the frame of interest is present is a sectionhaving a section length equal to or larger than the minimum limit numberof frames N.

When it is judged in step S101 that frames having a variation crossingthe threshold TH3 are continuously present by the minimum limit numberof frames N or more immediately before, immediately after, or before andafter the frame of interest, the processing proceeds to step S102. Thedetermining unit 164 determines, for example, the high displayrate/normal resolution type V13 among the display types for displayingan image using the main line data as a display type for the frame ofinterest. The processing proceeds to step S93 in FIG. 29. Thereafter,the processing described above is performed.

Concerning the frames in the mixed section in which frames having avariation crossing the threshold TH3 are continuously present by theminimum limit number of frames N or more, a display type for displayingan image using (image data serving as) the main line data or a displaytype for displaying an image using (image data serving as) the proxydata is determined as a display type for all the frames. This makes itpossible to prevent seek from being frequently performed.

Thus, in FIG. 30, for example, the high display rate/normal resolutiontype V13 among the display types for displaying an image using the mainline data is determined as all display types for the frames in the mixedsection in which frames having a variation crossing the threshold TH3are continuously present by the minimum limit number of frames N ormore.

Besides, it is possible to determine, for example, the normal type V12among the display types for displaying an image using the main line dataas display types of the frames in the mixed section in which frameshaving a variation crossing the threshold TH3 are continuously presentby the minimum limit number of frames N or more. It is also possible todetermine, for example, the ultrahigh display rate/low resolution typeV14 or the ultra-ultrahigh display rate/low resolution type V15 fordisplaying an image using the proxy data as display types for frames inthe mixed section in which frames having a variation crossing thethreshold TH3 are continuously present by the minimum limit number offrames N or more.

On the other hand, when it is judged in step S101 that frames having avariation crossing the threshold TH3 are not continuously present by theminimum limit number of frames N or more immediately before, immediatelyafter, and before and after the frame of interest, i.e., when a mixedsection in which the frame of interest is present (a mixed section ofinterest) is a section having a section length smaller than the minimumlimit number of frames N, the processing proceeds to step S103. Thecontinuity judging unit 163 judges whether one of a variation in a frameadjacent to the mixed section of interest among temporally precedingframes on the left side of the mixed section of interest having asection length smaller than the minimum limit number of frames N (a leftframe) and a variation in a frame adjacent to the mixed section ofinterest among temporally following frames on the right side of themixed section of interest (a right frame) is equal to or larger than thethreshold TH3 and the other is smaller than the threshold TH3. In otherwords, the continuity judging unit 163 judges whether the mixed sectionof interest having a section length smaller than the minimum limitnumber of frames N is a section between a large variation section and asmall variation section.

When it is judged in step S103 that the mixed section of interest havinga section length smaller than the minimum limit number of frames N is asection between a large variation section and a small variation section,the processing proceeds to step S102. As described above, thedetermining unit 164 determines the high display rate/normal resolutiontype V13 as a display type for the frame of interest. The processingproceeds to step S93 in FIG. 29. Thereafter, the processing describedabove is performed.

Both the large variation section and the small variation section on bothsides of the mixed section of interest having a section length smallerthan the minimum limit number of frames N are sections having a sectionlength equal to or larger than the minimum limit number of frames N.Moreover, the display type (the ultrahigh display rate/low resolutiontype V14 or the ultra-ultrahigh display/low resolution type V15) fordisplaying an image using the proxy data is determined as a display typefor frames in the large variation section. The display type (the stillimage type V11, the normal type V12, or the high display rate/normalresolution type V13) for displaying an image using the main line data isdetermined as a display type for frames in the small variation section.

Concerning the frames in the mixed section of interest having a sectionlength smaller than the minimum limit number of frames N between a largevariation section and a small variation section, a display type isidentical with a display type for frames in the large variation sectionor a display type for frames in the small variation section. This makesit possible to prevent seek from being frequently performed.

Thus, in FIG. 30, for example, the high display rate/normal resolutiontype V13 among the display types for displaying an image using the mainline data is determined as all display types for frames in the mixedsection of interest in which frames having a variation crossing thethreshold TH3 are less than the minimum limit number of frames N andthat is between a large variation section and a small variation section.

Besides, it is possible to determine, for example, the normal type V12among the display types for displaying an image using the main line dataor the ultrahigh display rate/low resolution type V14 among the displaytypes for displaying an image using the proxy data as the display typesfor frames in the mixed section of interest in which frames having avariation crossing the threshold TH3 are less than the minimum limitnumber of frames N and that is between a large variation section and asmall variation section.

On the other hand, when it is judged in step S103 that the mixed sectionof interest having a section length smaller than the minimum limitnumber of frames N is not a section between a large variation sectionand a small variation section, the processing proceeds to step S104. Thecontinuity judging unit 163 judges whether both a variation in a frameadjacent to the mixed section of interest among temporally precedingframes on the left side of the mixed section of interest having asection length smaller than the minimum limit number of frames N (a leftframe) and a variation in a frame adjacent to the mixed section ofinterest among temporally following frames on the right side of themixed section of interest (a right frame) are equal to or larger thanthe threshold TH3. In other words, the continuity judging unit 163judges whether the mixed section of interest having a section lengthsmaller than the minimum limit number of frames N is a section betweenlarge variation sections.

When it is judged in step S104 that the mixed section of interest havinga section length smaller than the minimum limit number of frames N isnot a section between large variation sections, the processing proceedsto step S102. As described above, the determining unit 164 determines,for example, the high display rate/normal resolution type V13 among thedisplay types for displaying an image using the main line data as adisplay type for the frame of interest. The processing proceeds to stepS93 in FIG. 29. Thereafter, the processing described above is performed.

In other words, concerning the frames in the mixed section of interesthaving a section length smaller than the minimum limit number of framesN between two small variation sections, a display type is identical withthe display type for displaying an image using the main line datadetermined as a display type for frames in the two small variationsections. This makes it possible to prevent seek from being frequentlyperformed.

On the other hand, when it is judged in step S104 that the mixed sectionof interest having a section length smaller than the minimum limitnumber of frames N is a section between large variation sections, theprocessing proceeds to step S105. The determining unit 164 determines,for example, the ultrahigh display rate/low resolution type V14 amongthe display types for displaying an image using the proxy data as adisplay type for the frame of interest. The processing proceeds to stepS93 in FIG. 29. Thereafter, the processing described above is performed.

In other words, concerning the frames in the mixed section of interesthaving a section length smaller than the minimum limit number of framesN between two large variation sections, a display type is identical withthe display type for displaying an image using the proxy data determinedas a display type for frames in the two large variation sections. Thismakes it possible to prevent seek from being frequently performed.

FIG. 31 is a diagram showing another example of display types determinedby the display-type determining unit 93 in FIG. 6.

In FIG. 31, as the display types, there are three types, namely, a lowresolution/normal display rate type C1, a normal type C2, and a normalresolution/low display rate type C3 in which resolutions and displayrates in displaying a frame are different.

Concerning frames of the low resolution/normal display rate type C1, animage is displayed at a display rate (a normal rate) identical with aframe rate of a moving image and with a resolution identical with thatof the image data serving as the proxy data (a resolution lower thanthat of the image data serving as the main line data).

Concerning frames of the normal type C2, an image is displayed at adisplay rate of a normal rate and with a resolution identical with thatof the image data serving as the main line data (a normal resolution).

Concerning frames of the normal resolution/low display rate type C3, animage is displayed at a display rate lower than the normal rate, forexample, a display rate one half of the normal rate and with aresolution identical with that of the image data serving as the mainline data (a resolution higher than that of the image data serving asthe proxy data).

When there are three types, namely, the low resolution/normal displayrate type C1, the normal type C2, and the normal resolution/low displayrate type C3 as the display types as shown in FIG. 31, the variationcalculating unit 92 (FIG. 6) calculates fineness informationrepresenting a degree of spatial change in a frame as a variation. Thedisplay-type determining unit 93 compares, for example, a variation(fineness information) of each of frames and two predeterminedthresholds. The display-type determining unit 93 determines, on thebasis of a result of the comparison, the low resolution/normal displayrate type C1, the normal type C2, or the normal resolution/low displayrate type C3 as a display type for the frame.

FIG. 32 is a graph of fineness information serving as a variation and adisplay type in a unit of frame determined by the display-typedetermining unit 93 (FIG. 6) in a unit of frame on the basis of thevariation.

In FIG. 32, the abscissa indicates a frame and the ordinate indicates avariation.

The display-type determining unit 93 compares the variation (thefineness information) in a unit of frame with one of the threshold L andthe threshold H that are in a relation of L<H. The display-typedetermining unit 93 determines the normal resolution/low display ratetype C3, which is a kind of a high resolution/low display rate type, asa display type for frames having fineness information equal to or largerthan the threshold H, i.e., frames in which fineness informationindicates that a degree of spatial change is large.

When, for example, display of frames having a large degree of spatialchange, i.e., frames of an image having a large change in a pixel value(a complicated image) is performed with a low resolution in scrubperformed in edition, a spatial change in an image tends to beoverlooked. Thus, it is desirable that the frames of the complicatedimage are displayed with a high resolution.

However, burdens of processing for displaying a complicated image with ahigh resolution are larger than burdens of processing for displaying animage that is not complicated, for example, a flat image.

Thus, although the frames of the complicated image are displayed with ahigh resolution, instead, the normal resolution/low display rate typeC3, which is a kind of the high resolution/low display rate type fordisplaying an image at a low display rate, is adopted. When an image isdisplayed at a low display rate, compared with the case in which animage is displayed at a high display rate, burdens of processing aresmall. Thus, it is possible to reduce burdens on the apparatus for scrubwhile preventing a spatial change in an image from being overlooked inedition by displaying, for the frames of the complicated image, an imagehaving a high resolution at a low display rate.

On the other hand, the display-type determining unit 93 determines thelow resolution/normal display rate type C1 for displaying an image witha resolution lower than that of the normal resolution/low display ratetype C3 and at a display rate of the normal rate higher than that of thenormal resolution/low display rate type C3 as a display type for frameshaving fineness information smaller than the threshold L, i.e., framesin which fineness information indicates that a degree of spatial changeis small.

Concerning frames in which a degree of a spatial change is small, i.e.,frames of a flat image, even if an image is displayed with a highresolution, details of the image do not look so different from detailsof an image displayed with a low resolution.

When the details of the image do not look so different when the image isdisplayed with a high resolution and displayed with a low resolution inthis way, if the image is displayed with a high resolution, so to speak,excess burdens are imposed on the apparatus for scrub.

Thus, the low resolution/normal display rate type C1 for displaying animage with a resolution lower than that of the normal resolution/lowdisplay rate type C3 and at a display rate of the normal rate higherthan that of the normal resolution/low display rate type C3 isdetermined as a display type for frames of a flat image. This makes itpossible to prevent excess burdens from being imposed on the apparatusfor scrub.

The display-type determining unit 93 determines the normal type C2 as adisplay type for frames other than frames having fineness informationequal to or larger than the threshold H and frames having finenessinformation smaller than the threshold L, i.e., frames having finenessinformation equal to or larger than the threshold L and smaller than thethreshold H.

In other words, the normal type C2 for displaying an image at the normalrate (a display rate identical with the frame rate of the moving image)and with the normal resolution as a display type for frames of an imagethat is not so complicated and not flat.

Processing of the display-type determining unit 93 in FIG. 22 in thecase in which the low resolution/normal display rate type C1, the normaltype C2, or the normal resolution/low display rate type C3 shown in FIG.31 is determined as a display type on the basis of fineness informationserving as a variation will be explained with reference to flowcharts inFIGS. 33 and 34.

When a variation in a unit of frame is supplied from the variationcalculating unit 92 (FIG. 6), in step S121, the display-type determiningunit 93 causes the storing unit 161 (FIG. 22) to cache (temporarilystore) the variation in a unit of frame. The processing proceeds to stepS122.

In FIGS. 33 and 34, fineness information is supplied from the variationcalculating unit 92 to the display-type determining unit 93 as avariation.

In step S122, the threshold processing unit 162 performs thresholdprocessing for comparing the variation (the fineness information) in aunit of frame stored in the storing unit 161 and the threshold H or thethreshold L. The threshold processing unit 162 supplies comparisoninformation representing a result of the comparison of the variation ina unit of frame and the threshold H or the threshold L to the continuityjudging unit 163 in a unit of frame. The processing proceeds to stepS123.

In step S123, the continuity judging unit 163 selects a front frame in atime series order not set as a frame of interest yet among framesforming a moving image, for which an Fy file is about to be created bythe Fy-file creating unit 76 (FIG. 6), as a frame of interest. Theprocessing proceeds to step S124.

In step S124, the continuity judging unit 163 judges, on the basis ofthe comparison information from the threshold processing unit 162,whether a variation in the frame of interest is smaller than thethreshold L and frames having a variation smaller than the threshold Lare continuously present by the minimum limit number of frames N or moreimmediately before, immediately after, or before and after the frame ofinterest.

When it is judged in step S124 that the variation in the frame ofinterest is smaller than the threshold L and frames having a variationsmaller than the threshold L are continuously present by a minimum limitnumber of frames N or more immediately before, immediately after, beforeand after the frame of interest, the continuity judging unit 163supplies judgment information representing a result of the judgment tothe determining unit 164. The processing proceeds to step S125.

In step S125, the determining unit 164 determines, on the basis of thejudgment information from the continuity judging unit 163, the lowresolution/normal display rate type C1 as a display type for the frameof interest. The processing proceeds to step S130.

According to steps S124 and S125, when the variation in the frame ofinterest is smaller than the threshold L and, moreover, frames having avariation smaller than the threshold L are continuously present by thenumber of frames N or more immediately before, immediately after, orbefore and after the frame of interest, the low resolution/normaldisplay rate type C1 is determined as a display type for the frame ofinterest. This determination is based on the following reason.

When a display type is determined out of the low resolution/normaldisplay rate type C1, the normal type C2, and the normal resolution/lowdisplay rate type C3 shown in FIG. 31, in scrub, an image having a lowresolution is displayed for frames of the low resolution/normal displayrate type C1. In other words, an image is displayed by processing proxydata having a smaller amount of data (than that of main line data).

On the other hand, concerning frames of the normal type C2 and thenormal resolution/low display rate type C3, an image having a highresolution is displayed. In other words, an image is displayed byprocessing image data of main line data having a larger amount of data(than that of proxy data).

For example, when the optical disk 7 (FIG. 1) is a professional disc inwhich main line data and proxy data are recorded, scrub is performedusing the main line data or the proxy data recorded in the optical disk7. Then, as in the case explained with reference to FIG. 23, in framesin time series forming a moving image, when frames of the lowresolution/normal display rate type C1 for displaying an image using theproxy data and frames of the normal type C2 or the normal resolution/lowdisplay rate type C3 for displaying an image using the main line dataare frequently switched, seek is frequently performed. Thus, it may bedifficult to smoothly perform display of a frame designated by operationof the scrub bar 53 (FIG. 3).

In order to prevent seek from being frequently performed, it isnecessary to prevent frames of the low resolution/normal display ratetype C1 for displaying an image using the proxy data and frames of thenormal type C2 or the normal resolution/low display rate type C3 fordisplaying an image using the main line data from being frequentlyswitched.

On the other hand, in both frames of the normal type C2 and frames ofthe normal resolution/low display rate type C3, an image is displayedusing the main line data. Thus, seek is not a problem concerningswitching of the frames of the normal type C2 and the frames of thenormal resolution/low display rate type C3.

Thus, in FIGS. 33 and 34, switching of frames of a display type fordisplaying an image using the proxy data (the low resolution/normaldisplay rate type C1) and frames of a display type for displaying animage using the main line data (the normal type C2 or the normalresolution/low display rate type C3) is prevented from being caused in asection of frames continuing by at least the minimum limit number offrames N. For this purpose, in steps S124 and S125, when a variation inthe frame of interest is smaller than the threshold L and frames havinga variation smaller than the threshold L continue by the minimum limitnumber of frames N or more immediately before, immediately after, orbefore and after the frame of interest, the low resolution/normaldisplay rate type C1 for displaying an image using the proxy data isdetermined as a display type for the frame of interest.

Further, switching of frames of a display type for displaying an imageusing the proxy data and frames of a display type for displaying animage using the main line data is prevented from being caused in asection of frames continuing by at least the minimum limit number offrames N. For this purpose, in steps S126 to S129 described later, whena variation in the frame of interest is equal to or larger than thethreshold L and frames having a variation equal to or larger than thethreshold L continues by the minimum limit number of frames N or moreimmediately before, immediately after, or before and after the frame ofinterest, the normal type C2 or the normal resolution/low display ratetype C3 for displaying an image using the main line data is determinedas a display type for the frame of interest.

When it is judged in step S124 that the variation in the frame ofinterest is not smaller than the threshold L or, even if the variationin the frame of interest is smaller than the threshold L, frames havinga variation smaller than the threshold L are not continuously present bythe minimum limit number of frames N or more immediately before,immediately after, and before and after the frame of interest, theprocessing proceeds to step S126. The continuity judging unit 163judges, on the basis of the comparison information from the thresholdprocessing unit 162, whether a variation in the frame of interest isequal to or larger than the threshold L and frames having a variationequal to or larger than the threshold L are continuously present by theminimum limit number of frames N or more immediately before, immediatelyafter, or before and after the frame of interest.

When it is judged in step S126 that the variation in the frame ofinterest is equal to or larger than the threshold L and frames having avariation equal to or larger than the threshold L are continuouslypresent by the minimum limit number of frames N or more immediatelybefore, immediately after, or before and after the frame of interest,the continuity judging unit 163 supplies judgment informationrepresenting a result of the judgment to the determining unit 164. Theprocessing proceeds to step S127.

The determining unit 164 receives, from the continuity judging unit 163,supply of the judgment information representing the result of thejudgment that the variation in the frame of interest is equal to orlarger than the threshold L and frames having a variation equal to orlarger than the threshold L are continuously present by the minimumlimit number of frames N or more immediately before, immediately after,or before and after the frame of interest. Then, in step S127, thedetermining unit 164 judges, on the basis of the comparison informationfrom the threshold processing unit 162, whether a variation in the frameof interest is equal to or larger than the threshold H.

When it is judged in step S127 that the variation in the frame ofinterest is equal to or larger than the threshold H, the processingproceeds to step S128. The determining unit 164 determines the normalresolution/low display rate type C3 as a display type for the frame ofinterest. The processing proceeds to step S130.

When it is judged in step S127 that the variation in the frame ofinterest is not equal to or larger than the threshold H, i.e., when thevariation in the frame of interest is equal to or larger than thethreshold L and smaller than the threshold H, the processing proceeds tostep S129. The determining unit 164 determines the normal type C2 as adisplay type for the frame of interest. The processing proceeds to stepS130.

According to steps S126 to S129, when the variation in the frame ofinterest is equal to or larger than the threshold L and frames having avariation equal to or larger than the threshold L are continuouslypresent by the number of frames N or more immediately before,immediately after, or before and after the frame of interest, the normalresolution/low display rate type C3 or the normal type C2 for displayingan image using the main line data is determined as a display type forthe frame of interest. This is for the purpose of preventing seek frombeing frequently performed at the time of scrub as described above.

In step S130, the continuity judging unit 163 judges whether there is aframe not selected as a frame of interest yet among frames forming amoving image, for which an Fy file is about to be created by the Fy-filecreating unit 76 (FIG. 6).

When it is judged in step S130 that there is a frame not selected as aframe of interest yet, the processing returns to step S123. The framenot selected as a frame of interest yet is selected as a frame ofinterest anew. The same processing is repeated.

When it is judged in step S130 that there isn't a frame not selected asa frame of interest yet, the display-type determining unit 93 ends theprocessing.

On the other hand, when it is judged in step S126 that the variation inthe frame of interest is not equal to or larger than the threshold L or,even if the variation in the frame of interest is equal to or largerthan the threshold L, frames having a variation equal to or larger thanthe threshold L are not continuously present by the minimum limit numberof frames N or more immediately before, immediately after, and beforeand after the frame of interest, the processing proceeds to step S151 inFIG. 34. The continuity judging unit 163 judges whether frames having avariation crossing the threshold L are continuously present by theminimum limit number of frames N or more immediately before, immediatelyafter, or before and after the frame of interest.

When a variation in the frame of interest is smaller than the thresholdL and frames having a variation smaller than the threshold L arecontinuously present by the number of frames N or more immediatelybefore, immediately after, or before and after the frame of interest, instep S125 in FIG. 33, the low resolution/normal display rate type C1 isdetermined as a display type for the frame of interest.

When a variation in the frame of interest is equal to or larger than thethreshold L and frames having a variation equal to or larger than thethreshold L are continuously present by the minimum limit number offrames N or more immediately before, immediately after, or before andafter the frame of interest, in steps S128 or S129 in FIG. 33, thenormal resolution/low display rate type C3 or the normal type C2 isdetermined as a display type for the frame of interest.

Therefore, the processing in step S151 in FIG. 34 is performed whenframes having a variation smaller than the threshold L are notcontinuously present by the number of frames N or more and frames havinga variation equal to or larger than the threshold L are not continuouslypresent by the minimum limit number of frames N or more immediatelybefore, immediately after, and before and after the frame of interest.This is a case in which the frame of interest is a frame in a section inwhich frames having a variation crossing the threshold L are present (asection in which frames having a variation smaller than the threshold Land frames having a variation equal to or larger than the threshold Lare mixed) and a section in which the number of continuing frames havinga variation smaller than the threshold L and the number of continuingframes having a variation equal to or larger than the threshold L aresmaller than the minimum limit number of frames N at most (hereinafteralso referred to as mixed section as appropriate).

As in the case explained with reference to FIG. 25, the mixed section istypically present between a section in which frames having a variationsmaller than the threshold L are continuously present by the number offrames N or more (hereinafter also referred to as small variationsection as appropriate) and a section in which frames having a variationequal to or larger than the threshold L are continuously present by theminimum limit number of frames N or more (hereinafter also referred tolarge variation section as appropriate), between two large variationsections, or between two small variation sections.

Consequently, the mixed section can be divided into a section having asection length (the number of frames) equal to or larger than theminimum limit number of frames N and a section having a section lengthsmaller than the minimum limit number of frames N.

The mixed section can also be divided into a section between a largevariation section and a small variation section, a section between largevariation sections, and a section between small variation sections.

In step S151, the continuity judging unit 163 judges whether frameshaving a variation crossing the threshold L are continuously present bythe minimum limit number of frames N or more immediately before,immediately after, or before and after the frame of interest, i.e., amixed section in which the frame of interest is present is a sectionhaving a section length equal to or larger than the minimum limit numberof frames N.

When it is judged in step S151 that frames having a variation crossingthe threshold L are continuously present by the minimum limit number offrames N or more immediately before, immediately after, or before andafter the frame of interest, the processing proceeds to step S152. Thedetermining unit 164 determines, for example, the normal type C2 amongthe display types for displaying an image using the main line data as adisplay type for the frame of interest. The processing proceeds to stepS130 in FIG. 33. Thereafter, the processing described above isperformed.

Concerning the frames in the mixed section in which frames having avariation crossing the threshold L are continuously present by theminimum limit number of frames N or more, a display type for displayingan image using (image data serving as) the main line data or a displaytype for displaying an image using (image data serving as) the proxydata is determined as a display type for all the frames. This makes itpossible to prevent seek from being frequently performed.

Thus, in FIG. 34, for example, the normal type C2 among the displaytypes for displaying an image using the main line data is determined asall display types for the frames in the mixed section in which frameshaving a variation crossing the threshold L are continuously present bythe minimum limit number of frames N or more.

Besides, it is possible to determine, for example, the lowresolution/normal display rate type C1 for displaying an image using theproxy data as display types of the frames in the mixed section in whichframes having a variation crossing the threshold L are continuouslypresent by the minimum limit number of frames N or more.

On the other hand, when it is judged in step S151 that frames having avariation crossing the threshold L are not continuously present by theminimum limit number of frames N or more immediately before, immediatelyafter, and before and after the frame of interest, i.e., when a mixedsection in which the frame of interest is present (a mixed section ofinterest) is a section having a section length smaller than the minimumlimit number of frames N, the processing proceeds to step S153. Thecontinuity judging unit 163 judges whether one of a variation in a frameadjacent to the mixed section of interest among temporally precedingframes on the left side of the mixed section of interest having asection length smaller than the minimum limit number of frames N (a leftframe) and a variation in a frame adjacent to the mixed section ofinterest among temporally following frames on the right side of themixed section of interest (a right frame) is smaller than the thresholdL and the other is equal to or larger than the threshold L. In otherwords, the continuity judging unit 163 judges whether the mixed sectionof interest having a section length smaller than the minimum limitnumber of frames N is a section between a large variation section and asmall variation section.

When it is judged in step S153 that the mixed section of interest havinga section length smaller than the minimum limit number of frames N is asection between a large variation section and a small variation section,the processing proceeds to step S152. As described above, thedetermining unit 164 determines the normal type C2 as a display type forthe frame of interest. The processing proceeds to step S130 in FIG. 33.Thereafter, the processing described above is performed.

Both the large variation section and the small variation section on bothsides of the mixed section of interest having a section length smallerthan the minimum limit number of frames N are sections having a sectionlength equal to or larger than the minimum limit number of frames N.Moreover, the display type (the normal type C2 or the normalresolution/low display rate type C3) for displaying an image using themain line data is determined as a display type for frames in the largevariation section. The display type (the low resolution/normal displayrate type C1) for displaying an image using the proxy data is determinedas a display type for frames in the small variation section.

Concerning the frames in the mixed section of interest having a sectionlength smaller than the minimum limit number of frames N between a largevariation section and a small variation section, a display type isidentical with a display type for frames in the large variation sectionor a display type for frames in the small variation section. This makesit possible to prevent seek from being frequently performed.

Thus, in FIG. 34, for example, the normal type C2 among the displaytypes for displaying an image using the main line data is determined asall display types for frames in the mixed section of interest in whichframes having a variation crossing the threshold L are less than theminimum limit number of frames N and that is between a large variationsection and a small variation section.

Besides, it is possible to determine, for example, a display type forframes in a small variation section, i.e., the low resolution/normaldisplay rate type C1 for displaying an image using the proxy data as thedisplay types for frames in the mixed section of interest in whichframes having a variation crossing the threshold L are less than theminimum limit number of frames N and that is between a large variationsection and a small variation section.

On the other hand, when it is judged in step S153 that the mixed sectionof interest having a section length smaller than the minimum limitnumber of frames N is not a section between a large variation sectionand a small variation section, the processing proceeds to step S154. Thecontinuity judging unit 163 judges whether both a variation in a frameadjacent to the mixed section of interest among temporally precedingframes on the left side of the mixed section of interest having asection length smaller than the minimum limit number of frames N (a leftframe) and a variation in a frame adjacent to the mixed section ofinterest among temporally following frames on the right side of themixed section of interest (a right frame) are smaller than the thresholdL. In other words, the continuity judging unit 163 judges whether themixed section of interest having a section length smaller than theminimum limit number of frames N is a section between small variationsections.

When it is judged in step S154 that the mixed section of interest havinga section length smaller than the minimum limit number of frames N isnot a section between large variation sections, i.e., when the mixedsection of interest having a section length smaller than the minimumlimit number of frames N is a section between large variation sections,the processing proceeds to step S152. As described above, thedetermining unit 164 determines a display type for frames in the largevariation section, i.e., for example, the normal type C2 among thedisplay types for displaying an image using the main line data as adisplay type for the frame of interest. The processing proceeds to stepS130 in FIG. 33. Thereafter, the processing described above isperformed.

In other words, concerning the frames in the mixed section of interesthaving a section length smaller than the minimum limit number of framesN between two small variation sections, a display type is identical withthe display type for displaying an image using the main line datadetermined as a display type for frames in the two small variationsections. This makes it possible to prevent seek from being frequentlyperformed.

On the other hand, when it is judged in step S154 that the mixed sectionof interest having a section length smaller than the minimum limitnumber of frames N is a section between small variation sections, theprocessing proceeds to step S155. The determining unit 164 determines,for example, the low resolution/normal display rate type C1 fordisplaying an image using the proxy data as a display type for the frameof interest. The processing proceeds to step S130 in FIG. 33.Thereafter, the processing described above is performed.

In other words, concerning the frames in the mixed section of interesthaving a section length smaller than the minimum limit number of framesN between two large variation sections, a display type is identical withthe low resolution/normal display rate type C1 for displaying an imageusing the proxy data determined as a display type for frames in the twolarge variation sections. This makes it possible to prevent seek frombeing frequently performed.

FIG. 35 is a diagram showing an example of an Fy file created by thefile creating unit 95 in FIG. 6 when motion information and finenessinformation are calculated as variations by the variation calculatingunit 92 in FIG. 6, a display type based on the motion information and adisplay type based on the fineness information are determined by thedisplay-type determining unit 93, and both the variations and thedisplay types are selected by the selecting unit 94.

In the Fy file in FIG. 35, from the left to the right, a frame numberindicating a place of a frame from the top, a time code of a framecorresponding to the frame number, motion information and finenessinformation serving as variation in the frames corresponding to theframe number, and a display type based on the motion information and adisplay type based on the fineness information of the framecorresponding to the frame number are sequentially arranged.

FIG. 36 is a diagram showing an example of an Fy file created by thefile creating unit 95 in FIG. 6 when motion information and finenessinformation are calculated as variations by the variation calculatingunit 92 in FIG. 6 and only the variations are selected by the selectingunit 94.

In the Fy file in FIG. 36, from the left to the right, a frame numberindicating a place of a frame from the top, a time code of a framecorresponding to the frame number, and motion information and finenessinformation serving as variation in the frame corresponding to the framenumber are sequentially arranged.

FIG. 37 is a diagram showing an example of an Fy file created by thefile creating unit 95 in FIG. 6 when motion information and finenessinformation are calculated as variations by the variation calculatingunit 92 in FIG. 6, a display type based on the motion information and adisplay type based on the fineness information are determined by thedisplay-type determining unit 93, and only the display types areselected by the selecting unit 94.

In the Fy file in FIG. 37, from the left to the right, a frame numberindicating a place of a frame from the top, a time code of a framecorresponding to the frame number, and a display type based on motioninformation and a display type based on fineness information of theframe corresponding to the frame number are sequentially arranged.

In the cases described above, the display types are determined on thebasis of the motion information or the fineness information. However, itis also possible to determine the display types on the basis of both themotion information and the fineness information.

FIG. 38 is a diagram showing another example of display types determinedby the display-type determining unit 93 in FIG. 6.

In FIG. 38, as the display types, there are four types, namely, a stillimage type VC1 representing display in a still image and a normal typeVC2, a high display rate/low resolution type VC3, and a normal displayrate/low resolution type VC4 in which resolutions in displaying a frameand display rates in displaying a frame are different.

Concerning frames of the still image type VC1, like the still image typeV1 in FIG. 20, even if a frame to be displayed is changed, as long as animage of the frame to be displayed can be regarded as an image identicalwith (an image of) a frame displayed immediately before the image, (theimage) of the frame displayed immediately before the image is displayed(continued to be displayed).

Concerning frames of the normal type VC2, like the normal type V2 inFIG. 20, an image is displayed at a display rate identical with a framerate of a moving image (a normal rate) and with a resolution identicalwith that of image data serving as main line data (a normal resolution).

Concerning frames of the high display rate/low resolution type VC3, likethe high display rate/low resolution type V3 in FIG. 20, an image isdisplayed at a display rate higher than the normal rate, for example, adisplay rate twice as high as the normal rate and with a resolutionidentical with that of image data serving as proxy data (a resolutionlower than that of image data serving as main line data).

Concerning frames of the normal display rate/low resolution type VC4, animage is displayed at a display rate of the normal rate and with aresolution identical with that of the image data serving as the proxydata (a resolution lower than the image data serving as the main linedata).

The high display rate/low resolution type VC3 and the normal displayrate/low resolution type VC4 are only different in that, whereas animage is displayed at a display rate higher than the normal rate in thehigh display rate/low resolution type VC3, an image is displayed at (adisplay rate of) the normal rate in the normal display rate/lowresolution type VC4.

When there are the four types, namely, the still image type VC1, thenormal type VC2, the high display rate/low resolution type VC3, and thenormal display rate/low resolution type VC4 as the display types, asshown in FIG. 38, the variation calculating unit 92 (FIG. 6) calculatesmotion information and fineness information as variations. On the basisof both the motion information and the fineness information, forexample, the display-type determining unit 93 (FIG. 6) compares themotion information and two thresholds and compares the finenessinformation and one threshold. The display-type determining unit 93determines, on the basis of a result of the comparisons, the still imagetype VC1, the normal type VC2, the high display rate/low resolution typeVC3, or the normal display rate/low resolution type VC4 as a displaytype for a frame.

FIG. 39 is a graph of motion information and fineness informationserving as variations in a unit of frame and a display type determinedby the display-type determining unit 93 (FIG. 6) in a unit of frame onthe basis of the motion information and the fineness information.

A graph in the upper part in FIG. 39 represents motion information in aunit of frame and a graph in the lower part in FIG. 39 representsfineness information in a unit of frame.

In FIG. 39, the abscissa indicates a frame and the ordinate indicatesvariations (motion information and fineness information).

The display-type determining unit 93 (FIG. 6) compares motioninformation in a unit of frame with one of the threshold L and thethreshold H that are in a relation of L<H. The display-type determiningunit 93 determines, so to speak, a provisional display type(provisionally determines a display type) on the basis of a result ofthe comparison.

For example, in the same manner as the case explained with reference toFIGS. 20 to 25, the display-type determining unit 93 determines, on thebasis of motion information of a frame, the still image type V1, thenormal type V2, or the high display rate/low resolution type V3 shown inFIG. 20 as a provisional display type for the frame (provisionallydetermines a display type).

Consequently, as explained with reference to FIGS. 20 to 25, roughlyspeaking, the still image type V1, the normal type V2, and the highdisplay rate/low resolution type V3 are provisionally determined as adisplay type for frames having motion information smaller than thethreshold L, a display type for frames having motion information equalto or larger than the threshold L and smaller than the threshold H, anda display type for frames having motion information equal to or largerthan the threshold H, respectively.

The display-type determining unit 93 (FIG. 6) compares the finenessinformation in a unit of frame with a predetermined threshold K. Thedisplay-type determining unit 93 finally determines the still image typeVC1, the normal type VC2, the high display rate/low resolution type VC3,or the normal display rate/low resolution type VC4 shown in FIG. 38 as adisplay type for the frames on the basis of a result of the comparisonand the provisional display type provisionally determined on the basisof the motion information.

The display-type determining unit 93 finally determines the still imagetype VC1 and the high display rate/low resolution type VC3 as a displaytype for the frames, a provisional display type for which is the stillimage type V1, and a display type for the frames, a provisional displaytype for which is the high display rate/low resolution type V3,respectively.

The display-type determining unit 93 finally determines the normal typeVC2 as a display type for frames having fineness information equal to orlarger than the threshold K, i.e., frames of an image not regarded as aflat image among frames, a provisional display type for which is thenormal type V2. Moreover, the display-type determining unit 93 finallydetermines the normal display rate/low resolution type VC4 as a displaytype for frames having fineness information smaller than the thresholdK, i.e., frames of a flat image among the frames, a provisional displaytype for which is the normal type V2.

The frames, a provisional display type for which is the normal type V2,are frames that have a certain degree of motion, although not very largemotion. The frames having fineness information smaller than thethreshold K are frames of a flat image. Therefore, among the frames, aprovisional display type for which is the normal type V2, frames havingfineness information smaller than the threshold K are frames of a flatimage, although having a certain degree of motion.

Even if an image has a certain degree of motion, as long as the image isflat, when the image is displayed with a low resolution, a spatialchange in the image does not tend to be overlooked. When the image isdisplayed with a low resolution in scrub, it is possible to reduceburdens imposed on the apparatus for scrub.

Thus, concerning the frames, a provisional display type for which is thenormal type V2, having a certain degree of motion, in principle, thenormal type VC2 for displaying an image with the normal resolution isdetermined as a display type. However, as long as fineness informationis smaller than the threshold K, the normal display rate/low resolutiontype VC4 for displaying an image with a resolution lower than the normalresolution is finally determined as the display type.

In FIG. 39, the normal display rate/low resolution type VC4 is finallydetermined as a display type for a frame in a section D_(p) that hasfineness information smaller than the threshold K, a provisional displaytype based on motion information of which is the normal type V2.

The still image type VC1, the normal type VC2, the high display rate/lowresolution type VC3, or the normal display rate/low resolution type VC4shown in FIG. 38 is finally determined as a display type on the basis ofmotion information and fineness information in a unit of frame. Thisdisplay type is hereinafter referred to as a final display type asappropriate.

Processing of the display-type determining unit 93 in FIG. 22 in thecase in which the still image type VC1, the normal type VC2, the highdisplay rate/low resolution type VC3, or the normal display rate/lowresolution type VC4 shown in FIG. 38 is determined as a display type (afinal display type) on the basis of both motion information and finenessinformation will be explained with reference to a flowchart in FIG. 40.

The variation calculating unit 92 (FIG. 6) calculates motion informationand fineness information as variations and supplies the motioninformation and the fineness information to the display-type determiningunit 93.

In step S181, the display-type determining unit 93 determines, on thebasis of the motion information in a unit of frame from the variationcalculating unit 92 (FIG. 6), a display type (a provisional displaytype) based on the motion information as explained with reference toFIGS. 23 and 24. The processing proceeds to step S182.

In step S182, the display-type determining unit 93 causes the storingunit 161 (FIG. 22) to cache the fineness information in a unit of framesupplied from the variation calculating unit 92. The processing proceedsto step S183.

In step S183, the threshold processing unit 162 (FIG. 22) performsthreshold processing for comparing the fineness information in a unit offrame stored in the storing unit 161 and the threshold K. The thresholdprocessing unit 162 supplies comparison information representing aresult of the comparison of the fineness information in a unit of frameand the threshold K to the continuity judging unit 163 in a unit offrame. The processing proceeds to step S184.

In step S184, the continuity judging unit 163 (FIG. 22) selects a frontframe in a time series order not set as a frame of interest yet amongframes forming a moving image, for which an Fy file is about to becreated by the Fy-file creating unit 76 (FIG. 6), as a frame ofinterest. The continuity judging unit 163 supplies information on theframe of interest to the determining unit 164 (FIG. 22). The processingproceeds to step S185.

In step S185, the determining unit 164 judges whether a display typebased on motion information of the frame of interest (the provisionaldisplay type determined in step S181) is the still image type V1.

When it is judged in step S185 that the display type based on the motioninformation of the frame of interest is the still image type V1, theprocessing proceeds to step S186. The determining unit 164 finallydetermines the still image type VC1 as a final display type for theframe of interest (a display type based on motion information andfineness information of the frame of interest). The processing proceedsto step S193.

When it is judged in step S185 that the display type based on the motioninformation of the frame of interest is not the still image type V1, theprocessing proceeds to step S187. The determining unit 164 judgeswhether the display type based on the motion information of the frame ofinterest is the high display rate/low resolution type V3.

When it is judged in step S187 that the display type based on the motioninformation of the frame of interest is the high display rate/lowresolution type V3, the processing proceeds to step S188. Thedetermining unit 164 finally determines the high display rate/lowresolution type VC3 as a final display type for the frame of interest.The processing proceeds to step S193.

When it is judged in step S187 that the display type based on the motioninformation of the frame of interest is not the high display rate/lowresolution type V3, i.e., when the display type based on the motioninformation of the frame of interest is the normal type V2, theprocessing proceeds to step S189. The continuity judging unit 163 (FIG.22) judges, on the basis of the comparison information from thethreshold processing unit 162, whether fineness information of the frameof interest, a display type based on the motion information of which isthe normal type V2, is smaller than the threshold K and frames havingfineness information smaller than the threshold K are continuouslypresent by the minimum limit number of frames N or more immediatelybefore, immediately after, or before and after the frame of interest.

When it is judged in step S189 that the fineness information of theframe of interest is not smaller than the threshold K or, even if thefineness information of the frame of interest is smaller than thethreshold K, frames having fineness information are not continuouslypresent by the minimum limit number of frames N or more immediatelybefore, immediately after, and before and after the frame of interest,the continuity judging unit 163 supplies judgment informationrepresenting a result of the judgment to the determining unit 164 (FIG.22). The processing proceeds to step S190.

The determining unit 164 (FIG. 22) receives, from the continuity judgingunit 163, the supply of the judgment information representing the resultof the judgment that the fineness information of the frame of interestis not smaller than the threshold K and frames having finenessinformation smaller than the threshold K are not continuously present bythe minimum limit number of frames N or more immediately before,immediately after, and before and after the frame of interest. Then, instep S190, the determining unit 164 finally determines the normal typeVC2 as a final display type for the frame of interest, a display typebased on the motion information of which is the normal type V2. Theprocessing proceeds to step S193.

When it is judged in step S189 that the fineness information of theframe of interest is smaller than the threshold K and frames havingfineness information smaller than the threshold K are continuouslypresent by the minimum limit number of frames N or more immediatelybefore, immediately after, or before and after the frame of interest,the processing proceeds to step S191. The continuity judging unit 163(FIG. 22) judges whether there is a section in which the number ofcontinuing frames having an identical type is less than the minimumlimit number of frames N when a display type for frames, a display typebased on the motion information of which is the normal type V2, amongframes of a section of the frames continuing by the minimum limit numberof frames N or more immediately before, immediately after, or before andafter the frame of interest having the fineness information smaller thanthe threshold K is changed to the normal display rate/low resolutiontype VC4.

For example, in FIG. 39, the section D_(p) is a section in which frameshaving fineness information smaller than the threshold K continue by theminimum limit number of frames N or more. In this case, in step S191, itis judged whether there is a section in which the number of continuingframes having an identical type is less than the minimum limit number offrames N among sections in which the frames of the identical displaytype continue in a time series of display types when a display type forframes, a display type based on the motion information of which is thenormal type V2, among frames of the section D_(p) is changed to thenormal display rate/low resolution type VC4. The display types are basedon the motion information and fineness information finally obtained fora moving image, for which an Fy file is about to be created.

When it is judged in step S191 that, when the display type for frames, adisplay type based on the motion information of which is the normal typeV2, among the frames in the section of frames continuing by the minimumlimit number of frames N or more immediately before, immediately after,or before and after the frame of interest having fineness informationsmaller than the threshold K is changed to the normal display rate/lowresolution type VC4, there is such a section in which the number ofcontinuing frames having an identical type is less than the minimumlimit number of frames N, i.e., when it is likely that seek isfrequently performed if the final display type of the frame of interest,a display type based on the motion information of which is the normaltype V2, is changed to the normal display rate/low resolution type VC4,the processing proceeds to step S190. As described above, thedetermining unit 164 (FIG. 22) finally determines the normal type VC2 asthe final display type for the frame of interest, a display type basedon the motion information of which is the normal type V2. The processingproceeds to step S193.

When it is judged in step S191 that, when the display type for frames, adisplay type based on the motion information of which is the normal typeV2, among the frames in the section of frames continuing by the minimumlimit number of frames N or more immediately before, immediately after,or before and after the frame of interest having fineness informationsmaller than the threshold K is changed to the normal display rate/lowresolution type VC4, there is no such section in which the number ofcontinuing frames having an identical type is less than the minimumlimit number of frames N, the processing proceeds to step S192. Thedetermining unit 164 (FIG. 22) finally determines the normal displayrate/low resolution type VC4 as the final display type for the frame ofinterest, a display type based on the motion information of which is thenormal type V2. The processing proceeds to step S193.

When the display type (the final display type) is determined out of thestill image type VC1, the normal type VC2, the high display rate/lowresolution type VC3, and the normal display rate/low resolution type VC4shown in FIG. 38, in scrub, concerning frames of the still image typeVC1 and frames of the normal type VC2, an image having a high resolution(an image having a normal resolution) is displayed. In other words, animage is displayed by processing image data of the main line data havinga large amount of data (than that of the proxy data).

Concerning frames of the high display rate/low resolution type VC3 andframes of the normal display rate/low resolution type VC4, an imagehaving a low resolution is displayed. In other words, an image isdisplayed by processing image data of the proxy data having a smalleramount of data (than that of the main line data).

For example, when the optical disk 7 (FIG. 1) is a professional disc inwhich main line data and proxy data are recorded, scrub is performedusing the main line data or the proxy data recorded in the optical disk7. Then, as in the case explained with reference to FIG. 23, in framesin time series forming a moving image, when frames of the high displayrate/low resolution type VC3 or the normal display rate/low resolutiontype VC4 for displaying an image using the proxy data and frames of thestill image type VC1 or the normal type VC2 for displaying an imageusing the main line data are frequently switched, seek is frequentlyperformed. Thus, it may be difficult to smoothly perform display of aframe designated by operation of the scrub bar 53 (FIG. 3).

In order to prevent seek from being frequently performed, it isnecessary to prevent frames of the high display rate/low resolution typeVC3 or the normal display rate/low resolution type VC4 for displaying animage using the proxy data and frames of the still image type VC1 or thenormal type VC2 for displaying an image using the main line data frombeing frequently switched.

Therefore, in FIG. 40, in steps S190 to S192, the normal displayrate/low resolution type VC4 is determined as the final display type forthe frame of interest, a display type based on the motion information ofwhich is the normal type V2 only in the following case. That is, thereis no section in which the number of continuing frames having anidentical type is less than the minimum limit number of frames N even ifthe display type for frames, a display type based on the motioninformation of which is the normal type V2, among the frames in thesection of frames continuing by the minimum limit number of frames N ormore immediately before, immediately after, or before and after theframe of interest having fineness information smaller than the thresholdK is changed to the normal display rate/low resolution type VC4. Inother case (when there is a section in which the number of continuingframes having the identical type (the final display type) is less thanthe minimum limit number of frames N), the normal type V2 is determinedas the final display type for the frame of interest, a display typebased on the motion information of which is the normal type VC2.

In step S193, the continuity judging unit 163 judges whether there is aframe not selected as a frame of interest yet among frames forming amoving image, for which an Fy file is about to be created by the Fy-filecreating unit 76 (FIG. 6).

When it is judged in step S193 that there is a frame not selected as aframe of interest yet, the processing returns to step S184. The framenot selected as a frame of interest yet is selected as a frame ofinterest anew. The same processing is repeated.

When it is judged in step S193 that there isn't a frame not selected asa frame of interest yet, the display-type determining unit 93 ends theprocessing.

FIG. 41 is a diagram showing an example of an Fy file created by thefile creating unit 95 in FIG. 6 when motion information and finenessinformation are calculated as variations by the variation calculatingunit 92 in FIG. 6, a display type (a final display type) based on themotion information and the fineness information is determined by thedisplay-type determining unit 93, and both the variations and thedisplay type are selected by the selecting unit 94.

In the Fy file in FIG. 41, from the left to the right, a frame numberindicating a place of a frame from the top, a time code of a framecorresponding to the frame number, motion information and finenessinformation serving as variation in the frame corresponding to the framenumber, and a display type (a final display type) based on the motioninformation and the fineness information of the frame corresponding tothe frame number are sequentially arranged.

FIG. 42 is a diagram showing an example of a structure of the drive 5 inFIG. 1.

The drive 5 includes a recording unit/reproducing unit 181, a decoder182, an Fy-file creating unit 183, and an external I/F 184.

The recording unit/reproducing unit 181 writes (records) to the opticaldisk 7 (a file of) data supplied from the external I/F 184 and an Fyfile supplied from the Fy-file creating unit 183. The recordingunit/reproducing unit 181 reads out data from the optical disk 7 andsupplies the data to the decoder 182 or the external I/F 184.

The decoder 182 decodes the data supplied from the recordingunit/reproducing unit 181 and supplies the data to the Fy-file creatingunit 183.

The Fy-file creating unit 183 has the same structure as the Fy-filecreating unit 76 in FIG. 4. The Fy-file creating unit 183 creates an Fyfile serving as a metafile from the data supplied from the decoder 182and supplies the Fy file to the recording unit/reproducing unit 181.

The external I/F 184 receives the data supplied from the recordingunit/reproducing unit 181 and outputs the data to the outside. Theexternal I/F 184 receives data supplied from the outside and suppliesthe data to the recording unit/reproducing unit 181.

In the drive 5 constituted as described above, the recordingunit/reproducing unit 181 records data supplied via the external I/F184, for example, according to operation by the user in the optical disk7. The recording unit/reproducing unit 181 reads out data recorded inthe optical disk 7 and outputs the data to the outside via the externalI/F 184.

For example, when the optical disk 7 is a professional disc in which AVfiles having main line data and proxy data stored therein and metafileshaving metadata stored therein are recorded, in the drive 5, therecording unit/reproducing unit 181 checks whether an Fy file is presentin the metafile recorded in the optical disk 7, for example, when theoptical disk 7 is inserted into the drive 5 or according to operation bythe user.

When an Fy file is not recorded in the optical disk 7, the recordingunit/reproducing unit 181 reads out image data serving as the main linedata from the optical disk 7 to acquire the image data and supplies theimage data to the decoder 182.

The image data serving as the main line data recorded in the opticaldisk 7 is encoded, for example, in the MPEG2 system as explained withreference to FIG. 5. Thus, the decoder 182 decodes the main line datafrom the recording unit/reproducing unit 181 in the MPEG2 system andsupplies image data obtained as a result of the decoding to the Fy-filecreating unit 183.

The Fy-file creating unit 183 creates an Fy file serving as the metafilehaving the metadata of the image data (the image data recorded in theoptical disk 7) from the decoder 182 in the same manner as the Fy-filecreating unit 76 in FIG. 4.

The Fy-file creating unit 183 calculates a variation in each of framesof the image data from the decoder 182. Moreover, the Fy-file creatingunit 183 determines a display type on the basis of the variation in theframe if necessary. The Fy-file creating unit 183 creates an Fy filehaving stored therein one or both of the variation and the display typein a unit of frame and supplies the Fy file to the recordingunit/reproducing unit 181.

The recording unit/reproducing unit 181 records the Fy file suppliedfrom the Fy-file creating unit 183 in the optical disk 7.

In the drive 5 in FIG. 42, for example, an encoded bit stream obtainedby encoding image data in the MPEG2 system is supplied to the externalI/F 184 from the outside and the recording unit/reproducing unit 181records (a file of) the encoded bit stream supplied from the externalI/F 184 in the optical disk 7. In this case, the Fy-file creating unit183 can create an Fy file as described above after the recording of theencoded bit stream ends. Alternatively, the recording unit/reproducingunit 181 supplies the encoded bit stream supplied from the external I/F184 to the decoder 182 and causes the decoder 182 to decode the encodedbit stream and supply the bit stream decoded to the Fy-file creatingunit 183. Consequently, the Fy-file creating unit 183 can create an Fyfile in parallel to the recording of the encoded bit stream in theoptical disk 7.

A kind of a pixel value used for calculating a variation in the frame isnot specifically limited. When the pixel value includes a luminancesignal (Y) and a color difference signal (Cb, Cr), it is possible tocalculate a variation using the luminance signal or using the colordifference signal. However, since a change in the luminance signal has alarge influence on the human visual sense is larger than a change in thecolor difference signal, it is desirable to calculate a variation usingthe luminance signal.

When the pixel value includes color components of R, G, and B, forexample, it is possible to calculate a variation using a square sum orthe like of the color components of R, G, and B.

As the motion information serving as the variation, it is possible toadopt values that quantitatively represent intensity of motion otherthan the values explained with reference to FIGS. 11 to 14. Similarly,as the fineness information serving as the variation, it is possible toadopt values (e.g., values called difficulty and flatness) thatquantitatively represent fineness of an image other than the valuesexplained with reference to FIGS. 15 to 18.

For example, when a variation is calculated using image data encoded inthe MPEG2 system as explained with reference to FIG. 42, it is possibleto analyze (parse) an encoded bit stream obtained by encoding the imagedata in the MPEG2 system and calculate a variation using a result of theanalysis.

A motion vector is included in the encoded bit stream. Thus, when themotion information serving as the variation is calculated using themotion vector as explained with reference to FIGS. 11 and 12, it ispossible to analyze the encoded bit stream to obtain the motion vectorincluded in the encoded bit stream as a result of the analysis andcalculate motion information using the motion vector.

DCT factors obtained by subjecting a block of 8×8 pixels to DCTconversion are included in the encoded bit stream. Thus, when finenessinformation serving as the variation is calculated using the DCT factorsas explained with reference to FIGS. 15 and 16, it is possible toanalyze the encoded bit stream to obtain the DCT factors included in theencoded bit stream as a result of the analysis and calculate finenessinformation using the DCT factors.

Moreover, a so-called DC component, which is a DCT factor on the upperleft, among the DCT factors obtained by subjecting the block of 8×8pixels to DCT conversion is an average of pixels values of the 8×8pixels of the block. Thus, when the fineness information serving as thevariation is calculated using the average of the pixel values of theblock as explained with reference to FIGS. 17 and 18, it is possible toanalyze the encoded bit stream to obtain the DC component among the DCTfactors included in the encoded bit stream as a result of the analysisand calculate fineness information using the DC component, i.e., theaverage of the pixel values of the block.

FIG. 43 is a diagram showing an example of a functional structure of thePC 1 functioning as an editing system when the CPU 22 (FIG. 2) of the PC1 in FIG. 1 executes the editing program.

The editing system includes the keyboard 2, the mouse 3, the display 41,an edition control unit 201, a stream-memory control unit 221, and astream memory 222.

The edition control unit 201 includes a GUI control unit 211, astream-decoding-position detecting unit 212, a decoder-supply controlunit 213, an Fy-file managing unit 214, a display-type acquiring unit215, a decoder 216, an Fy-file creating unit 217, and a display controlunit 218.

The GUI control unit 211 controls the display control unit 218 to causethe display 41 (FIG. 2) to display a GUI such as the viewer 51 shown inFIG. 3. The GUI control unit 211 receives an operation signal suppliedfrom the keyboard 2 or the mouse 3 according to operation of thekeyboard 2 or the mouse 3 by the user on the GUI displayed on thedisplay 41. The GUI control unit 211 supplies necessary information andthe like to the stream-decoding-position detecting unit 212, thedata-supply control unit 213, the display control unit 218, and the likeaccording to the operation signal.

Specifically, the GUI control unit 211 supplies, for example, accordingto the operation signal supplied from the keyboard 2 or the mouse 3, acommand for requesting reproduction of a frame to thestream-decoding-position detecting unit 212 together with informationdesignating a stream (an AV file) of main line data or proxy data to besubjected to scrub and information designating a frame that should bereproduced in scrub.

Moreover, the GUI control unit 211 supplies thresholds used indetermining a display type on the basis of a variation to thedata-supply control unit 213 according to the operation signal suppliedfrom the keyboard 2 or the mouse 3.

The stream-decoding-position detecting unit 212 generates, according tothe command from the GUI control unit 211, a frame number serving asinformation specifying a frame to be displayed on the display 41 and astream number serving as information specifying a stream including data(main line data or proxy data) of the frame and supplies the framenumber and the stream number to the data-supply control unit 213.

The data-supply control unit 213 performs relay and the like for dataexchanged among blocks constituting the editing system.

Specifically, the data-supply control unit 213 receives for example, thethresholds from the GUI control unit 211 and supplies the thresholds tothe Fy-file managing unit 214. The data-supply control unit 213receives, for example, the frame number and the stream number from thestream-decoding-position detecting unit 212 and supplies the framenumber and the stream number to the display-type acquiring unit 215 andthe stream-memory control unit 221. Moreover, the data-supply controlunit 213 receives, for example, the stream (the main line data or theproxy data) from the stream-memory control unit 221 and supplies thestream to the decoder 216. The data-supply control unit 213 acquires,for example, the Fy file read out from the optical disk 7 by receivingthe Fy file and supplies the Fy file to the Fy-file managing unit 214.Moreover, the data-supply control unit 213 receives the Fy file suppliedfrom the Fy-file creating unit 217, supplies the Fy file to the opticaldisk 7, and causes the optical disk 7 to record the Fy file.

The Fy-file managing unit 214 manages (stores) the Fy file supplied fromthe data-supply control unit 213. The Fy-file managing unit 214determines (again) a display type by comparing the variation stored inthe Fy file and the thresholds supplied from the data-supply controlunit 213.

The display-type acquiring unit 215 acquires a display type for a framespecified by the frame number (and a necessary stream number) suppliedfrom the data-supply control unit 213, i.e., a frame displayed on thedisplay 41 by referring to the Fy file managed by the Fy-file managingunit 214. The display-type acquiring unit 215 supplies the display typeto the decoder 216, the display control unit 218, and the othernecessary blocks constituting the editing system such as the GUI controlunit 211.

The decoder 216 includes a memory control unit 216A. The decoder 216decodes the stream (the main line data or the proxy data) supplied fromthe data-supply control unit 213 and supplies image data of a frame(image data of a base band) obtained as a result of the decoding to theFy-file creating unit 217 and the display control unit 218.

The decoder 216 decodes the stream while causing the RAM 24 (FIG. 2) tostore data necessary in decoding the stream. The memory control unit216A included in the decoder 216 controls reading of the data from andwriting of the data in the RAM 24. The decoder 216 does not performdecoding in some cases when a display type for a frame displayed on thedisplay 41 is a still image type. The decoder 216 recognizes that thedisplay type for the frame displayed on the display 41 is the stillimage type by referring to the display type supplied from thedisplay-type acquiring unit 215.

The Fy-file creating unit 217 has the same structure as the Fy-filecreating unit 76 shown in FIG. 6. The Fy-file creating unit 217 createsan Fy file from the image data supplied from the decoder 216 andsupplies the image data to the data-supply control unit 213.

The display control unit 218 causes the display 41 to display a GUIaccording to the information supplied from the GUI control unit 211 andthe Fy file and the like managed by the Fy-file managing unit 214. Thedisplay control unit 218 causes the display 41 to display an imagecorresponding to the image data of the frame supplied from the decoder216 with a display method represented by the display type for the framesupplied from the display-type acquiring unit 215. An imagecorresponding to data stored in a frame buffer 218A serving as a storagearea in a part of the RAM 24 (FIG. 2) is displayed on the display 41.The display control unit 218 writes image data in the frame buffer 218Ato cause the display 41 to display an image corresponding to the imagedata.

The stream-memory control unit 221 controls reading of the stream (themain line data or the proxy data) from and writing of the stream in theoptical disk 7 and the stream memory 222. The stream control unit 221reads out data (main line data or proxy data) of a stream necessary fordecoding a frame specified by the frame number and the stream numberfrom the data-supply control unit 213 and supplies the data to thedata-supply control unit 213 and, if necessary, to the stream memory222.

The stream memory 222 is a storage area in a part of the RAM 24 (FIG. 2)and stores the data of the stream supplied from the stream-memorycontrol unit 221. The stream-memory control unit 221 supplies the dataof the stream read out from the optical disk 7 to the stream memory 222and causes the stream memory 222 to store the data. The stream-memorycontrol unit 221 reads out the data of the stream already stored in thestream memory 222. In reading out the data of the stream from the streammemory 222, it is possible to read out the data at high speed comparedwith speed of readout from the optical disk 7.

In the editing system constituted as described above, the Fy file isread out from the optical disk 7, supplied to the Fy-file managing unit214 via the data-supply control unit 213, and stored therein.

On the other hand, when a frame designated as a frame displayed on thedisplay 41 in scrub is referred to as a frame of interest, thedisplay-type acquiring unit 215 acquires a display type for the frame ofinterest by referring to the Fy file managed (stored) by the Fy-filemanaging unit 214 and supplies the display type to the display controlunit 218.

The decoder 216 decodes the data read out from the optical disk 7 andsupplied to the decoder 216 via the data-supply control unit 213 intoimage data of the frame of interest and supplies the image data to thedisplay control unit 218. The display control unit 218 causes thedisplay 41 to display an image corresponding to the image data of theframe of interest with a display method represented by the display typefor the frame of interest.

As described above, one or both of a variation and a display type in aunit of frame are selected and stored in the Fy file. Thus, in the Fyfile, both the variation and the display type are stored, the variationis stored but the display type is not stored, or the display type isstored but the variation is not stored.

When the display type is stored in the Fy file, i.e., when both thevariation and the display type are stored or when the display type isstored but the variation is not stored, it is possible to cause thedisplay 41 to display the image corresponding to the image data of theframe of interest using the display type stored in the Fy file as it is.

When the variation is stored in the Fy file, i.e., when both thevariation and the display type are stored or when the variation isstored but the display type is not stored, the Fy-file managing unit 214can determine a display type in a unit of frame on the basis of thevariation in a unit of frame stored in the Fy file and cause the display41 to display the image corresponding to the image data of the frame ofinterest using the display type.

When an Fy file is not recorded in the optical disk 7, the Fy-filecreating unit 217 can create an Fy file and record the Fy file in theoptical disk 7.

In this explanation, in the editing system in FIG. 43, scrub isperformed with the data recorded in the optical disk 7 as an object.Besides, for example, it is possible to perform scrub with the datarecorded in, for example, the hard disk 4 in FIG. 1 and the hard disk 25in FIG. 2 as objects.

FIG. 44 is a diagram showing an example of structures of the Fy-filemanaging unit 214 and the display-type acquiring unit 215 in FIG. 43.

The Fy-file managing unit 214 includes a file storing unit 231, avariation acquiring unit 232, a display-type determining unit 233, and adisplay-type writing unit 234.

The file storing unit 231 stores (data stored in) the Fy file suppliedfrom the data-supply control unit 213 (FIG. 43).

The variation acquiring unit 232 acquires the variation in a unit offrame by reading out the variation from the Fy file stored in the filestoring unit 231 and supplies the variation to the display-typedetermining unit 233.

The display-type determining unit 233 includes a storing unit 241, athreshold processing unit 242, a continuity judging unit 243, and adetermining unit 244. The display-type determining unit 233 determines(again) a display type in a unit of frame on the basis of the variationin a unit of frame supplied from the variation acquiring unit 232 in thesame manner as the display-type determining unit 93 in FIG. 22 andsupplies the display type to the display-type writing unit 234.

In the display-type determining unit 233, the storing unit 241, thethreshold processing unit 242, the continuity judging unit 243, and thedetermining unit 244 have the same structures as the storing unit 161,the threshold processing unit 162, the continuity judging unit 163, andthe determining unit 164 of the display-type determining unit 93 in FIG.22, respectively.

Like the threshold processing unit 162, the threshold processing unit242 compares a variation and a threshold in a unit of frame. However,the threshold compared with the variation by the threshold processingunit 242 is supplied from the GUI control unit 211 in FIG. 43 to thethreshold processing unit 242 via the data-supply control unit 213.

The display-type writing unit 234 stores (writes) the display type in aunit of frame supplied from the Fy-file managing unit 214 in associationwith the frame number and the time code stored in the Fy file (FIG. 26)stored in the file storing unit 231.

The display-type acquiring unit 215 includes a display-type readout unit251. The display-type readout unit 251 acquires the display typeassociated with the frame number of the frame of interest supplied fromthe data-supply control unit 213 by reading out the display type fromthe Fy file managed by the Fy-file managing unit 214, i.e., the Fy filestored in the file storing unit 231 of the Fy-file managing unit 214.The display-type readout unit 251 supplies the display type to thedecoder 216, the display control unit 218, and the like in FIG. 43.

In scrub performed in the editing system in FIG. 43, a frame (a frame ofinterest) to be displayed on (the display section 52 of the viewer 51displayed on) the display 41 is designated according to, for example, aposition of the scrub dial 54 of the scrub bar 53 shown in FIG. 3.

The GUI control unit 211 in FIG. 43 causes the display control unit 218to display the scrub bar 53 and recognizes a frame designated by aposition of the scrub dial 54 of the scrub bar 53 as a frame ofinterest.

A relation between a position of the scrub dial 54 and a framedesignated by the position will be explained with reference to FIG. 45.

FIG. 45 is a diagram showing the scrub bar 53.

In the scrub bar 53, a movable range in the horizontal direction inwhich the scrub dial 54 is movable (a range in length of, for example,about 10 to 20 cm in the horizontal direction) is divided intosubsections in a number identical with the number of frames forming (astream of) a moving image set to be subjected to scrub (hereinafterreferred to as designated sections as appropriate).

In FIG. 45, the movable range is divided into F designated sections.

When length in the horizontal direction of the designated sections(width in dividing the movable range into designated sections) isreferred to as a section length, in FIG. 45, the movable range isdivided (equally divided) into designated sections having an identicalsection length.

In the respective designated sections dividing the movable range, framesforming the moving image set to be subjected to scrub are allocated intime series, for example, in a direction from the designated section onthe left side to the designated section on the right side. Therefore,when a designated section ith from the left in the movable range isreferred to as an ith designated section and a frame ith from the top ofthe frames forming the moving image set to be subjected to scrub isreferred to as an ith frame, the ith frame is allocated to the ithdesignated section.

The GUI control unit 211 detects a designated section in which the scrubdial 54 is located at timing corresponding to a display rate indicatedby a display type for a frame displayed on the display 41 immediatelybefore scrub dial 54 is moved. The GUI control unit 211 recognizes aframe allocated to the designated section as a frame of interest.

As display types, the three types, namely, the still image type V1, thenormal type V2, and the high display rate/low resolution type V3 shownin FIG. 20 are adopted. A display rate of the still image type V1 andthe normal type V2 is, for example, 30 frames/second. A display rate ofthe high display rate/low resolution type V3 is, for example, 60frames/second.

FIG. 46 is a diagram showing display methods in scrub of frames of thenormal type V2 and the high display rate/low resolution type V3.

When a display type for a frame displayed on the display 41 immediatelybefore the scrub dial 54 is moved is the still image type V1 or thenormal type V2 having the display rate of 30 frames/second, the GUIcontrol unit 211 detects a designated section in which the scrub dial 54is located at an interval of 1/30 second. The GUI control unit 211recognizes a frame allocated to the designated section as a frame ofinterest. Therefore, when the display type is the normal type V2, theframe (the designated frame) displayed on the display 41 is updated atthe interval of 1/30 second, i.e., a display rate of 30 frames/second.

When a display type for a frame displayed on the display 41 immediatelybefore the scrub dial 54 is moved is, for example, the high displayrate/low resolution type V3 having the display rate of 60 frames/second,the GUI control unit 211 detects a designated section in which the scrubdial 54 is located at an interval of 1/60 second. The GUI control unit211 recognizes a frame allocated to the designated section as a frame ofinterest. Therefore, when the display type is the high display rate/lowresolution type V3, the frame (the frame of interest) displayed on thedisplay 41 is updated at an interval of 1/60 second, i.e., a displayrate of 60 frames/second.

Consequently, when the scrub dial 54 is moved, for example, at speed formoving from the left to the right by length of two designated sectionsin 1/30 second, for example, in a normal section in which frames, adisplay type for which is the normal type V2, continue, every otherframes, i.e., an ith frame, an i+2th frame, an i+4th frame, and the likeare set as a frame of interest at an interval of 1/30 second. As aresult, an image is displayed in a missing frame state at double speedon the display 41.

For example, the scrub dial 54 is moved at the speed for moving from theleft to the right by the length of two designated sections in 1/30second as in the case described above, i.e., when the scrub dial 54 ismoved at speed for moving from the left to the right by length of onedesignated section in 1/60 second. In this case, in a high displayrate/low resolution section in which frames of the high display rate/lowresolution type V3 continue, each of the frames, i.e., an ith frame, ani+1th frame, an i+2th frame, and the like are set as a frame of interestat an interval of 1/60 second. As a result, an image is displayedwithout missing frame at double speed on the display 41.

When the scrub dial 54 is moved at high moving speed for performingdisplay (reproduction) at double speed, missing frame occurs in framesof some display types and does not occur in frames of other displaytypes. Concerning frames of the high display rate/low resolution type V3having a high display rate compared with the normal type V2, i.e.,frames having large motions, missing frame occurs less easily even ifthe scrub dial 54 is moved fast. This makes it possible to prevent atemporal change in an image from being overlooked for the frames havinglarge motion.

As described above, whereas the frame of interest is updated at aninterval of 1/30 second in the normal section, the frame of interest isupdated at an interval of 1/60 second, which is a half of that in thenormal section, in the high display rate/low resolution section.

Therefore, the decoder 216 (FIG. 43) needs to decode frames in the highdisplay rate/low resolution section (frames, a display type for which isthe high display rate/low resolution type V3) at speed twice as high asthat of frames in the normal section (frames, a display type for whichis the normal type V2). Moreover, the frames in the high displayrate/low resolution section are frames having large motion than that ofthe frames in the normal section. Thus, when a resolution is fixed, theframes in the high display rate/low resolution have a larger amount ofdata necessary for decoding by the decoder 216 than that of the framesin the normal section. Consequently, large loads are imposed on displayof the frames in the high display rate/low resolution section comparedwith display of the frames in the normal section.

Thus, in the editing system in FIG. 43, when the frames in the normalsection are displayed, image data serving as main line data is decoded.When the frames in the high display rate/low resolution section aredisplayed, image data serving as proxy data having a smaller amount ofdata than the main line data is decoded. In this way, the burdensimposed on display of the frames in the high display rate/low resolutionsection are reduced.

In the case described above, the scrub dial 54 is moved, for example, atthe speed for moving from the left to the right by the length of twodesignated sections in 1/30 second. Besides, for example, when the scrubdial 54 is moved at speed for moving from the left to the right bylength of one designated section in 1/30 second, in the normal sectionin which the frames, a display type for which is the normal type V2,continue, each of the frames is set as a frame of interest at aninterval of 1/30 second. As a result, an image is displayed at singlespeed without missing frame on the display 41.

When the scrub dial 54 is moved at speed for moving from the left to theright by length of one designated section in 1/30 second, the scrub dial54 is moved from the left to the right by ½ of the length of onedesignated section in 1/60 second. Thus, in the high display rate/lowresolution section in which frames, a display type for which is the highdisplay rate/low resolution type V3, continue, an identical frame is setas a frame of interest in 1/30 second. As a result, an image is alsodisplayed at single speed without missing frame on the display 41.

Display of frames, a display type for which is the still image type V1,will be explained with reference to FIG. 47.

In the editing system in FIG. 43, concerning frames in the still imagesection in which frames, a display type for which is the still imagetype V1, continue, even if a frame of interest (a frame to be displayed)is changed among the frames in the still image section, an image of aframe displayed on the display 41 immediately before the frame ofinterest is changed.

Concerning the frames in the still image section, as long as a frame inthe still image section is a frame of interest, an image of a frame setas a frame of interest first in the still image section continues to bedisplayed.

For example, as shown in the top part in FIG. 47, at the start of scrub,the scrub dial 54 designates a frame in a section other than the stillimage section (here, the normal section or the high display rate/lowresolution section) as a frame of interest. In this case, when the usermoves the scrub dial 54 from the left to the right, according to themovement, a frame located at the left end in the still image sectionamong the frames in the still image section (a frame earliest in thestill image section) is designated as a frame of interest first. Then,the decoder 216 (FIG. 43) decodes image data of the frame designated asthe frame of interest first. An image corresponding to the image data isdisplayed on the display 41.

Thereafter, as long as a frame in the still image section is designatedas a frame of interest, the image of the frame designated as the frameof interest first is displayed as an image of a present frame ofinterest. Therefore, as long as a frame in the still image section isdesignated as a frame of interest, the decoder 216 does not decode imagedata of the present frame of interest except the image data of the framedesignated as the frame of interest first.

For example, as shown in the second diagram from the top of FIG. 47, atthe start of scrub, the scrub dial 54 designates a frame in the stillimage section as a frame of interest. In this case, the decoder 216(FIG. 43) decodes image data of the frame designated as the frame ofinterest at the start of scrub. An image corresponding to the image datais displayed on the display 41.

Thereafter, as long as a frame in the still image section is designatedas a frame of interest, the image of the frame designated as the frameof interest at the start of scrub is displayed as an image of a presentframe of interest. Therefore, as long as a frame in the still imagesection is designated as a frame of interest, the decoder 216 does notdecode image data of the present frame of interest except the image dataof the frame designated as the frame of interest at the start of scrub.

Moreover, for example, as shown in the third diagram from the top (thediagram at the bottom) in FIG. 47, at the start of scrub, the scrub dial54 designates a frame in a section other than the still image section(here, the normal section or the high display rate/low resolutionsection) as a frame of interest. In this case, when the user moves thescrub dial 54 from the right to the left, according to the movement, aframe located at the right end in the still image section among theframes in the still image section (a frame latest in the still imagesection) is designated as a frame of interest first. Then, the decoder216 (FIG. 43) decodes image data of the frame designated as the frame ofinterest first. An image corresponding to the image data is displayed onthe display 41.

Thereafter, as long as a frame in the still image section is designatedas a frame of interest, the image of the frame designated as the frameof interest first is displayed as an image of a present frame ofinterest. Therefore, as long as a frame in the still image section isdesignated as a frame of interest, the decoder 216 does not decode imagedata of the present frame of interest except the image data of the framedesignated as the frame of interest first.

Processing of scrub performed in the editing system in FIG. 43 will beexplained with reference to FIG. 48.

For example, the user inserts the optical disk 7 (FIG. 1) havingrecorded therein AV files and Fy files into the drive 5 and operates thekeyboard 2 or the mouse 3 to start an editing program. Then, the CPU 22(FIG. 2) executes the editing program. Consequently, the PC 1 functionsas the editing system in FIG. 43.

Thereafter, when the user operates the keyboard 2 or the mouse 3 todesignate an AV file to be subjected to scrub, an operation signalcorresponding to the operation is supplied from the keyboard 2 or themouse 3 to the GUI control unit 211.

In step S211, the GUI control unit 211 recognizes the AV file to besubjected to scrub according to the operation signal from the keyboard 2or the mouse 3. The processing proceeds to step S212.

In step S212, the Fy-file managing unit 214 caches (temporarily stores)an Fy file corresponding to the AV file recognized as the object ofscrub by the GUI control unit 211. The processing proceeds to step S213.

The GUI control unit 211 controls the data-supply control unit 213 viathe stream-decoding-position detecting unit 212 to supply the Fy filecorresponding to the AV file recognized to be subjected to scrub. Thedata-supply control unit 213 reads out the Fy file from the optical disk7 in accordance with the control by the GUI control unit 211 andsupplies the Fy file to the Fy-file managing unit 214. In step S212, theFy-file managing unit 214 causes the file storing unit 231 (FIG. 44) tostore the Fy file supplied from the data-supply control unit 213 asdescribed above.

In step S213, the variation acquiring unit 232 (FIG. 44) of the Fy-filemanaging unit 214 judges whether a display type is present in the Fyfile stored in the file storing unit 231 in the immediately precedingstep S212.

When it is judged in step S213 that a display type is present in the Fyfile stored in the file storing unit 231, step S214 described later isskipped. The processing proceeds to step S215.

When it is judged in step S213 that a display type is not present in theFy file stored in the file storing unit 231, the variation acquiringunit 232 reads out a variation in a unit of frame from the Fy filestored in the file storing unit 231 and supplies the variation to thedisplay-type determining unit 233 (FIG. 44). The processing proceeds tostep S214.

At least motion information serving as a variation in a unit of frame isstored in the Fy file stored in the file storing unit 231.

In step S214, the display-type determining unit 233 determines a displaytype in a unit of frame on the basis of the variation in a unit of framesupplied from the variation acquiring unit 232 in the same manner as thedisplay-type determining unit 93 in FIG. 22 and supplies the displaytype to the display-type writing unit 234 (FIG. 44). The display-typewriting unit 234 writes the display type from the display-typedetermining unit 233 in the Fy file in the file storing unit 231. Theprocessing proceeds from step S214 to step S215.

In determining a display type in step S214, the threshold processingunit 242 (FIG. 44) of the display-type determining unit 233 compares thevariation and a threshold. As the threshold, a threshold identical withthe threshold used by the threshold processing unit 162 of thedisplay-type determining unit 93 in FIG. 22 is used.

When the processing proceeds from step S213 to step S215 or from stepS214 to step S215 as described above, in the editing system in FIG. 43,GUI display processing for displaying a GUI such as the viewer 51 inFIG. 3 on the display 41 is started. Details of the GUI displayprocessing are described later.

In step S215, the GUI control unit 211 judges whether the keyboard 2 orthe mouse 3 is operated to finish the editing program.

When it is judged in step S215 that the keyboard 2 or the mouse 3 is notoperated to finish the editing program, the processing proceeds to stepS216. The GUI control unit 211 judges whether the scrub dial 54 providedin the scrub bar 53 of the viewer 51 (FIG. 3), which is displayed by theGUI display processing started when the processing proceeds from stepS213 to S215 or from step S214 to step S215, is moved. In other words,the GUI control unit 211 judges whether the user has moved the scrubdial 54 by operating the keyboard 2 or the mouse 3.

When it is judged in step S216 that the scrub dial 54 is moved, thismeans that the user has operated the keyboard 2 or the mouse 3 to movethe scrub dial 54 and an operation signal corresponding to the operationis supplied from the keyboard 2 or the mouse 3 to the GUI control unit211. In this case, the GUI control unit 211 sets a frame allocated to adesignated section (FIG. 45) in which the scrub dial 54 is located as aframe of interest. The GUI control unit 211 supplies information on theframe of interest to the display-type acquiring unit 215 via thestream-decoding-position detecting unit 212 and the data-supply controlunit 213. The processing proceeds to step S217.

In step S217, the display-type acquiring unit 215 acquires a displaytype for the frame of interest by reading out the display type from thefile storing unit 231 (FIG. 44) of the Fy-file managing unit 214 on thebasis of the information on the frame of interest supplied from the GUIcontrol unit 211 via the stream-decoding-position detecting unit 212 andthe data-supply control unit 213. The display-type acquiring unit 215supplies the display type to the GUI control unit 211, the decoder 216,and the display control unit 218. Moreover, in step S217, the decoder216 and the display control unit 218 judge the display type for theframe of interest from the display-type acquiring unit 215. The decoder216 and the display control unit 218 perform display control processingfor displaying an image of the frame of interest (steps S218 to S223described later). The processing returns to step S215.

When it is judged in step S217 that the display type for the frame ofinterest is the still image type V1, the processing proceeds to stepS218. The display control unit 218 judges, with reference to the Fy filestored in the file storing unit 231 of the Fy-file managing unit 214,whether a display type for a frame set as a frame of interestimmediately before the frame of interest (hereinafter referred to asprevious frame as appropriate) is the still image type V1 and theprevious frame and the frame of interest are frames in an identicalstill image section.

When it is judged in step S218 that the previous frame and the frame ofinterest are not frames in an identical still image section, theprocessing proceeds to step S219. When the data-supply control unit 213reads out main line data of the frame of interest (and data necessaryfor decoding of the frame of interest) from the optical disk 7 andsupplies the main line data to the decoder 216, the decoder 216 acquires(receives) the main line data of the frame of interest. The processingproceeds to step S220.

In step S220, the decoder 216 decodes the main line data of the frame ofinterest acquired from the data-supply control unit 213. The decoder 216supplies image data obtained as a result of the decoding to the displaycontrol unit 218. The processing proceeds to step S221. In step S221,the display control unit 218 causes the frame buffer 218A to store theimage data of the frame of interest from the decoder 216. The processingproceeds to step S222. The display control unit 218 causes the display41 to display an image corresponding to the image data stored in theframe buffer 218A.

Therefore, as an image of the frame, a display type for which is thestill image type V1, the image corresponding to the image data servingas the main line data, i.e., an image having a higher resolution (thanan image corresponding to the image data serving as the proxy data) isdisplayed.

On the other hand, when it is judged in step S218 that the previousframe and the frame of interest are frames in an identical still imagesection, the processing proceeds to step S222. The display control unit218 causes the display 41 to display the image corresponding to theimage data, which is stored in the frame buffer 218A in step S221, as animage of the frame of interest.

When the previous frame and the frame of interest are frames in anidentical still image section, as an image of a frame in the still imagesection, an image of a frame set as a frame of interest first amongframes in the still image section is displayed. Therefore, in this case,since decoder 216 does not have to perform decoding, it is possible toreduce burdens imposed on the PC 1.

On the other hand, when it is judged in step S217 that a display typefor the frame of interest is the normal type V2, the processingsequentially proceeds to steps S219 to S222. As described above, animage of the frame of interest is displayed.

In step S219, when the data-supply control unit 213 reads out main linedata of the frame of interest from the optical disk 7 and supplies themain line data to the decoder 216, the decoder 216 acquires the mainline data of the frame of interest. The processing proceeds to stepS220.

In step S220, the decoder 216 decodes the main line data of the frame ofinterest acquired from the data-supply control unit 213. The decoder 216supplies image data obtained as a result of the decoding to the displaycontrol unit 218. The processing proceeds to step S221. In step S221,the display control unit 218 causes the frame buffer 218A to store theimage data of the frame of interest from the decoder 216. The processingproceeds to step S222. The display control unit 218 causes the display41 to display an image corresponding to the image data stored in theframe buffer 218A.

Therefore, as an image of a frame, a display type for which is thenormal type V2, the image corresponding to the image data serving as themain line data, i.e., an image having a higher resolution (than an imagecorresponding to the image data serving as the proxy data) is displayed.

On the other hand, when it is judged in step S217 that a display typefor the frame of interest is the high display rate/low resolution typeV3, the processing proceeds to step S233. When the data-supply controlunit 213 reads out proxy data of the frame of interest (and datanecessary for decoding of the frame of interest) from the optical disk 7and supplies the proxy data to the decoder 216, the decoder 216 acquiresthe proxy data of the frame of interest. The processing proceeds to stepS220.

In step S220, the decoder 216 decodes the proxy data of the frame ofinterest acquired from the data-supply control unit 213. The decoder 216supplies image data obtained from a result of the decoding to thedisplay control unit 218. The processing proceeds to step S221. In stepS221, the display control unit 218 causes the frame buffer 218A to storethe image data of the frame of interest from the decoder 216. Theprocessing proceeds to step S222. The display control unit 218 causesthe display 41 to display an image corresponding to the image datastored in the frame buffer 218A.

Therefore, as an image of a frame, a display type for which is the highdisplay rate/low resolution type V3, an image corresponding to the imagedata serving as the proxy data, i.e., an image having a lower resolution(than an image corresponding to the image data serving as the main linedata) is displayed.

On the other hand, when it is judged in step S216 that the scrub dial 54is not moved, the processing proceeds to step S222. The display controlunit 218 causes the display 41 to display the image corresponding to theimage data, which is stored in the frame buffer 218A in step S221, as animage of the frame of interest.

When the scrub dial 54 is not moved from the time when the judgment instep S216 was performed, the display control unit 218 causes the display41 to display the image corresponding to the image data stored in theframe buffer 218A, i.e., the image displayed immediately before thejudgment (the image displayed in the last step S222) again.

After the image of the frame of interest is displayed in step S222 asdescribed above, the processing proceeds to step S224. The GUI controlunit 211 judges the display type for the frame of interest supplied fromthe display-type acquiring unit 215 in step S217.

When it is judged in step S224 that the display type for the frame ofinterest is the high display rate/low resolution type V3, the processingproceeds to step S225. The GUI control unit 211 judges whether timecorresponding to the display rate of the high display rate/lowresolution type V3, for example, 1/60 second has elapsed after it isjudged in the last step S216 whether the scrub dial 54 is moved.

When it is judged in step S225 that 1/60 second has not elapsed, theprocessing returns to step S225.

When it is judged in step S225 that 1/60 second has elapsed, theprocessing returns to step S215. Thereafter, the processing describedabove is repeated.

Therefore, when a frame, a display type for which is the high displayrate/low resolution type V3, is a frame of interest, in step S216, theGUI control unit 211 judges whether the scrub dial 54 is moved at aperiod of 1/60 second corresponding to the high display rate/lowresolution type V3.

When it is judged that the scrub dial 54 is moved, the display controlunit 218 causes the display 41 to display an image corresponding toimage data obtained by decoding proxy data of a frame allocated to adesignated section (FIG. 45) in which the scrub dial 54 is located afterthe movement.

When it is judged that the scrub dial 54 is not moved, the displaycontrol unit 218 causes the display 41 to display the image displayed onthe display 41 immediately before the judgment.

As described above, concerning the frame, a display type for which isthe high display rate/low resolution type V3, an image is displayed onthe display 41 at 60 frames/second, which is a display rate of thedisplay type.

On the other hand, when it is judged in step S224 that the display typefor the frame of interest is the still image type V1 or the normal typeV2, the processing proceeds to step S226. The GUI control unit 211judges whether time corresponding to the display rate of the still imagetype V1 and the normal type V2, for example, 1/30 second has elapsedafter it is judged in the last step S216 whether the scrub dial 54 ismoved.

When it is judged in step S226 that 1/30 second has not elapsed, theprocessing returns to step S226.

When it is judged in step S226 that 1/30 second has elapsed, theprocessing returns to step S215. Thereafter, the processing describedabove is repeated.

Therefore, when a frame, a display type for which is the still imagetype V1 or the normal type V2, is a frame of interest, the GUI controlunit 211 judges in step S216 whether the scrub dial 54 is moved at aperiod of 1/30 second corresponding to the still image type V1 and thenormal type V2.

When it is judged that the scrub dial 54 is moved, the display controlunit 218 causes the display 41 to display an image corresponding toimage data obtained by decoding main line data of a frame allocated to adesignated section (FIG. 45) in which the scrub dial 54 after themovement is located (an image having a high resolution).

When it is judged that the scrub dial 54 is not moved, the displaycontrol unit 218 causes the display 41 to display the image displayed onthe display 41 immediately before the judgment again.

As described above, concerning the frames, display types of which arethe still image type V1 and the normal type V2, an image is displayed onthe display 41 at 30 frames/second, which is a display rate of thedisplay types.

On the other hand, when the keyboard 2 or the mouse 3 is operated tofinish the editing program in step S215, i.e., for example, when theuser has operated the keyboard 2 or the mouse 3 to finish the editingprogram and an operation signal corresponding to the operation issupplied from the keyboard 2 or the mouse 3 to the GUI control unit 211,the processing of scrub (the execution of the editing program) ends.

As described above, a display type for a frame of interest, which is aframe designated by the scrub dial 54, is acquired and an image of theframe of interest is displayed with a display method represented by thedisplay type for the frame of interest. Thus, it is possible toappropriately perform scrub.

When the frame, a display type for which is the still image type V1,i.e., a frame having no motion is a frame of interest, image data of aframe set as a frame of interest first in a still image section in whichframes of the still image type V1 including the frame of interestcontinue is obtained by decoding the main line data and an imagecorresponding to the image data is displayed. Thus, high-quality imagesof an identical frame are displayed. As a result, the user can easilycheck contents of the image. Moreover, as long as frames in an identicalstill image section are continuously set as a frame of interest, it isunnecessary to decode the main line data. Thus, it is possible to reduceburdens imposed on the PC 1 serving as the editing system.

When the frame, a display type for which is the high display rate/lowresolution type V3, i.e., a frame having large (intense) motion is aframe of interest, an image is displayed at a high display rate. Thus,the user can accurately check contents of an image having intensemotion.

Moreover, when the frame, a display type for which is the high displayrate/low resolution type V3, is a frame of interest, the proxy datahaving a smaller amount of data than the main line data is decoded andan image corresponding to image data as a result of the decoding isdisplayed. Thus, it is possible to reduce burdens imposed on the PC 1serving as the editing system and prevent the editing program fromfailing to respond (the PC 1 from hanging up) because of large loadsimposed on the PC 1.

When the frame, a display type for which is the normal type V2, i.e., aframe having a certain degree of motion, although not intense, is aframe of interest, an image having a display rate lower than that of theframe, a display type for which is the high display rate/low resolutiontype V3, but having a resolution higher than that of the frame, adisplay type for which is the high display rate/low resolution type V3,is displayed. The frame, a display type for which is the normal type V2,has smaller motion of an image (a change in a pixel value among frames)is smaller than that of the frame, a display type for which is the highdisplay rate/low resolution type V3. Thus, even if the frame, a displaytype for which is the normal type V2, is displayed at a display ratelower than that of the frame, a display type for which is the highdisplay rate/low resolution type V3, the user can accurately checkcontents of the image.

Moreover, the frame, a display type for which is the normal type V2, hassmaller motion of an image than that of the frame, a display type forwhich is the high display rate/low resolution type V3. An amount of dataof the main line data of the frame, a display type for which is thenormal type V2, is smaller than an amount of data of the main line dataof the frame, a display type for which is the high display rate/lowresolution type V3. Therefore, when the frame, a display type for whichis the normal type V2, is a frame of interest, even if the main linedata is decoded and an image having a high resolution corresponding toimage data obtained as a result of the decoding is displayed, loadsimposed on the PC 1 serving as the editing system are smaller than loadsimposed when the main line data of the frame, a display type for whichis the high display rate/low resolution type V3, is decoded.

As described above, in the editing system in FIG. 43, in scrub, aresolution and a display rate of an image are changed according to adisplay type rather than displaying all frames with an identical displaymethod. Moreover, only a necessary frame is decoded according to adisplay type rather than decoding all frames set as a frame of interest.This makes it possible to perform appropriate scrub.

In the processing of scrub in FIG. 48, the display types are the threetypes, namely, the still image type V1, the normal type V2, and the highdisplay rate/low resolution type V3 shown in FIG. 20 determined on thebasis of motion information. However, it is also possible to perform theprocessing of scrub even if the display types are, for example, the fivetypes, namely, the still image type V11, the normal type V12, the highdisplay rate/normal resolution type V13, the ultrahigh display rate/lowresolution type V14, and ultra-ultrahigh display rate/low resolutiontype V15 shown in FIG. 27. It is also possible to perform the processingof scrub even if the display types are, for example, the three types,namely, the low resolution/normal display rate type C1, the normal typeC2, and the normal resolution/low display rate type C3 shown in FIG. 31determined on the basis of fineness information.

In the case described above, the main line data of the image having ahigh resolution and the proxy data of the image having a low resolutionare recorded in the optical disk 7, i.e., the images having two kinds ofresolutions including identical contents are recorded in the opticaldisk 7, and the image having a high resolution and the image having alow resolution are displayed according to a display type for a frame.Besides, for example, it is possible to record images of three kinds ofresolutions, i.e., high, medium, and low resolutions in the optical disk7 and display the image having a high resolution, the image having amedium resolution, and an image having a low resolution according to adisplay type for a frame.

As described above, in the editing system in FIG. 43, when theprocessing of scrub in FIG. 48 proceeds from step S213 to step S215 orproceeds from step S214 to S215, the GUI display processing fordisplaying a GUI such as the viewer 51 in FIG. 3 on the display 41 isstarted.

As described above, one or both of a variation and a display type in aunit of frame are selected and stored in an Fy file. In the followingexplanation, at least a variation of a variation and a display type isstored in a Fy file.

When a variation is stored but a display type is not stored in a Fyfile, it is possible to determine a display type in the Fy managing unit214 (FIG. 43) on the basis of the variation. Thus, for the editingsystem in FIG. 43 including the Fy managing unit 214, the Fy file inwhich a display type is not stored but a variation is stored issubstantially equivalent to an Fy file in which both a display type anda variation are stored.

On the other hand, when a display type is stored but a variation is notstored in an Fy file, since it is difficult to calculate a variationfrom the display type, concerning the Fy file in which a display type isstored but a variation is not stored, processing for calculating avariation in processing explained below is not performed. However, evenin the file in which a variation is not stored, if the Fy-file creatingunit 217 re-creates an Fy file in which a variation is stored from datastored in an AV file corresponding to the Fy file, it is possible toperform the processing for calculating a variation in the processingexplained below using the Fy file re-created.

FIG. 49 is a diagram showing an example of an edition window 301 servingas a GUI displayed on the display 41 by the GUI display processing.

In the edition window 301 in FIG. 49, the viewer 51 explained withreference to FIG. 3 is arranged on the upper right thereof. An Fy-filedisplay section 311, which is an area of a horizontally rectangularshape, is provided in a lower part of the edition window 301.

A GUI representing information and the like stored in an Fy filecorresponding to an AV file to be subjected to scrub is displayed in theFy-file display section 311.

Motion information is stored as a variation in the Fy file correspondingto the AV file to be subjected to scrub. The motion information iscompared with the threshold L or the threshold H, for example, asexplained with reference to FIGS. 20 and 21. Consequently, the stillimage type V1, the normal type V2, or the high display rate/lowresolution type V3 is determined as a display type.

In the Fy-file display section 311 in FIG. 49, a graph quantitativelyrepresenting a variation stored in the AV file to be subjected to scrub,i.e., the Fy file managed by the Fy-file managing unit 214 (hereinafterreferred to as variation graph as appropriate) is displayed. In FIG. 49(in figures described later as well), the horizontal direction (left toright direction) of the Fy-file display section 311 indicates elapse oftime, i.e., a frame (a place of a frame from the top) and the verticaldirection thereof indicates a variation.

Moreover, in the Fy-file display section 311, a position indicator 312serving as position information representing a position of a frameallocated to a designated section, in which the scrub dial 54 of theviewer 51 is located, is displayed.

In scrub, a frame allocated to the designated section, in which thescrub dial 54 of the scrub bar 53 is located, is set as a frame ofinterest displayed on the display section 52 of the viewer 51. Theposition indicator 312 is a GUI of a segment extending in the verticaldirection that is displayed in a position corresponding to the frame ofinterest among frames in time series forming a moving imagecorresponding to the AV file to be subjected to scrub (a moving image tobe subjected to scrub). When the scrub dial 54 is moved in thehorizontal direction and a frame of interest is changed from the framethat is a frame of interest before the movement to another frame, theposition indicator 312 is also moved in the horizontal direction toindicate a position corresponding to the another frame.

A scale in the horizontal direction of an area of the Fy-file displaysection 311 in which the variation graph is displayed and a scale of themovable range of the scrub bar 53 correspond to each other.

If length in the horizontal direction of the area in which the variationgraph of the Fy-file display section 311 and length of the movable rangeof the scrub bar 53 are set to the same length by multiplying the lengthof the horizontal direction of the area of the Fy-file display section311, in which the variation graph is displayed, by a predeterminednumber, a position of the position indicator 312 displayed in theFy-file display section 311 and a position of the scrub dial 54 coincidewith each other.

When the length in the horizontal direction of the area of the Fy-filedisplay section 311 in which the variation graph is displayed and thelength of the movable range of the scrub bar 53 are set to the samelength as described above, a value of the variation graph in theposition of the scrub dial 54 is a variation in a frame allocated to adesignated section in which the scrub dial 54 is located.

In FIG. 49, in the Fy-file display section 311, threshold indicators313H and 313L serving as threshold information representing thresholdsused in determining a display type are also displayed on the basis ofthe variation stored in the Fy file managed by the Fy-file managing unit214.

The threshold indicators 313H and 313L are GUIs of segments extending inthe horizontal direction that are displayed in positions in the verticaldirection corresponding to thresholds used in determining a displaytype. Therefore, when the variation stored in the Fy file managed by theFy-file managing unit 214 is, for example, motion information asdescribed above and the motion information is compared with thethreshold L or the threshold H to determine the still image type V1, thenormal type V2, or the high display rate/low resolution type V3 as adisplay type, the threshold indicator 313H is displayed in, for example,a position (in the vertical direction) corresponding to the threshold Hand the threshold indicator 313L is displayed in a positioncorresponding to the threshold L.

The user can move the threshold indicators 313H and 313L serving asthreshold information in the vertical direction by operating thekeyboard 2 or the mouse 3 of the editing system (FIG. 43) (e.g., bydragging a cursor). This makes it possible to change a threshold usedfor determining a display type to a threshold represented by thethreshold information after the movement. When the threshold used fordetermining a display type is changed, the display-type determining unit233 of the file managing unit 214 (FIG. 44) compares the variationstored in the Fy file and the threshold after change and determines adisplay type again on the basis of a result of the comparison.Processing for determining a display type again using the thresholdafter change in this way is performed in GUI display processingdescribed later.

According to the Fy-file display section 311 described above, the usercan visually grasp the transition of the variation over the entiremoving image to be subjected to scrub according to the variation graph.When the variation is, for example, motion information, the user cangrasp the transition of a degree of motion over the entire moving imageto be subjected to scrub according to the variation graph. As a result,the user can grasp in advance, for example, that the moving image to besubjected to scrub is a moving image having motion as a whole or havinglittle motion or a moving image having intense motion in a part offrames.

The user can grasp a degree of motion of a frame of interest displayedin (the display section 52 of) the viewer 51 and frames near the frameof interest according to the variation graph and the position indicator312 indicating a position of the frame of interest.

The user can grasp with what kind of display type (display method) animage is displayed in the viewer 51 according to the variation graph,the position indicator 312 indicating a position of the frame ofinterest, and the threshold indicators 313H and 313L. Moreover, the usercan recognize how the Fy file is used (e.g., used for displaying animage in scrub) in the editing system (FIG. 43).

The GUI display processing will be explained with reference to aflowchart in FIG. 50.

In the GUI display processing, the GUI control unit 211 controls thedisplay control unit 218 to display the edition window 301 (FIG. 49).The display control unit 218 causes the display 41 to display theedition window 301. In step S251, the display control unit 218 reads outthe Fy file managed by the Fy-file managing unit 214 (FIG. 43), i.e.,the Fy file stored in the file storing unit 231 (FIG. 44) of the Fy-filemanaging unit 214 and causes the Fy-file display section 311 of theedition window 301 to display a variation graph on the basis of avariation stored in the Fy file. The processing proceeds to step S252.

In step S252, the display control unit 218 causes the Fy-file displaysection 311 of the edition window 301 to display, for example, thethreshold indicators 313H and 313L (FIG. 49) according to thresholdsused for determining the display type stored in the Fy file stored inthe file storing unit 231 (FIG. 44) of the Fy-file managing unit 214.

The thresholds used for determining the display type stored in the Fyfile stored in the file storing unit 231 (FIG. 44) of the Fy-filemanaging unit 214 immediately after the GUI processing is started arereferred to as default thresholds.

Assuming that there are the two thresholds H and L as the defaultthresholds, the threshold indicators 313H and 313L corresponding to thetwo thresholds H and L, respectively, are displayed.

When there is only one threshold as the default threshold, a thresholdindicator serving as one piece of threshold information corresponding tothe one threshold is displayed.

When there are three or more thresholds as the default thresholds,threshold indicators serving as pieces of threshold information in anumber identical with the number of the default thresholds aredisplayed.

Thereafter, the processing proceeds from step S252 to step S253. Thedisplay control unit 218 acquires information on a position of the scrubdial 54 in the edition window 301 (FIG. 49) displayed on the display 41and a frame allocated to a designated section in which the scrub dial 54is located, i.e., a frame of interest as well from the GUI control unit211 (receives the information on the frame of interest supplied from theGUI control unit 211). The processing proceeds to step S254.

In step S254, the display control unit 218 causes the Fy-file displaysection 311 of the edition window 301 to display the position indicator312 (FIG. 49) according to the information on the frame of interestacquired from the GUI control unit 211 in the immediately preceding stepS253. The display control unit 218 causes the Fy-file display section311 to display the position indicator 312 (FIG. 49) of a segment shapeextending in the vertical direction in a position in the horizontaldirection of the Fy-file display section 311 corresponding to the frame(the frame of interest) allocated to the designated section in which thescrub dial 54 is located.

The processing proceeds from step S254 to step S255. The GUI controlunit 211 judges whether the keyboard 2 or the mouse 3 is operated tofinish the editing program.

When it is judged in step S255 that the keyboard 2 or the mouse 3 is notoperated to finish the editing program, the processing proceeds to stepS256. The GUI control unit 211 judges whether the scrub dial 54 ismoved, i.e., the user has moved the scrub dial 54 by operating thekeyboard 2 or the mouse 3.

When it is judged in step S256 that the scrub dial 54 is moved, thismeans that the user has operated the keyboard 2 or the mouse 3 to movethe scrub dial 54 and an operation signal corresponding to the operationis supplied from the keyboard 2 or the mouse 3 to the GUI control unit211. In this case, the GUI control unit 211 sets a frame allocated to adesignated position (FIG. 45) in which the scrub dial 54 is located as aframe of interest and supplies information on the frame of interest tothe display control unit 218. The processing returns to step S253.Thereafter, the same processing is repeated.

When the scrub dial 54 is moved in the horizontal direction and theframe of interest is changed from the frame set as the frame of interestimmediately before the movement to another frame according to theprocessing in steps S253 to S256, the position indicator 312 is alsomoved in the horizontal direction to indicate a position correspondingto the another frame.

Steps S255 and S256 are identical with steps S215 and S216 in FIG. 48,respectively. Therefore, it is possible to perform the processing instep S215 and the processing in S216 in FIG. 48 as the processing instep S255 and the processing in S256, respectively. In this case, asexplained with reference to FIG. 48, when a display type for the frameof interest is the still image type V1 or the normal type V2, steps S255and S256 are performed at a period of 1/30 second. When a display typefor the frame of interest is the high display rate/low resolution typeV3, steps S255 and S256 are performed at a period of 1/60 second.

On the other hand, when it is judged in step S256 that the scrub dial 54is not moved, the processing proceeds to step S257. The GUI control unit211 judges whether movement of the threshold indicator 313H or 313L(FIG. 49) is started, i.e., the user has operated the keyboard 2 or themouse 3 to move the threshold indicator 313H or 313L.

When it is judged in step S257 that the movement of the thresholdindicator 313H or 313L is not started, the processing returns to stepS255. Thereafter, the same processing is repeated.

When it is judged in step S257 that the movement of the thresholdindicator 313H or 313L is started, this means that the user has operatedthe keyboard 2 or the mouse 3 to move the threshold indicator 313H or313L and an operation signal corresponding to the operation is suppliedfrom the keyboard 2 or the mouse 3 to the GUI control unit 211. In thiscase, the GUI control unit 211 calculates, according to the operationsignal from the keyboard 2 or the mouse 3, a position to which thethreshold indicator 313H or 313L is moved. The GUI control unit 211supplies information indicating the position to the display control unit218. The processing proceeds to step S258.

In step S258, the display control unit 218 causes the Fy-file displaysection 311 to display the threshold indicator 313H or 313L in aposition corresponding to the information from the GUI control unit 211instead of the position displayed immediately before the thresholdindicator 313H or 313L is moved. The processing proceeds to step S259.

In step S259, the GUI control unit 211 judges whether the movement ofthe threshold indicator 313H or 313L is finished, i.e., the user hasfinished the operation of the keyboard 2 or the mouse 3 for moving thethreshold indicator 313H or 313L.

When it is judged in step S259 that the operation for moving thethreshold indicator 313H or 313L is finished, this means that the usercontinues to operate the keyboard 2 or the mouse 3 to move the thresholdindicator 313H or 313L and an operation signal corresponding to theoperation is supplied from the keyboard 2 or the mouse 3 to the GUIcontrol unit 211. In this case, the GUI control unit 211 calculates,according to the operation signal from the keyboard 2 or the mouse 3, aposition to which the threshold indicator 313H or 313L is moved. The GUIcontrol unit 211 supplies information indicating the position to thedisplay control unit 218. The processing returns to step S258.Thereafter, the same processing is repeated.

According to the processing in steps S257 to S259, the thresholdindicator 313H or 313L is moved in the vertical direction in accordancewith operation by the user for moving the threshold indicator 313H or313L.

On the other hand, when it is judged in step S259 that the operation formoving the threshold indicator 313H or 313L is finished, this means thatthe user has finished the operation of the keyboard 2 or the mouse 3 formoving the threshold indicator 313H or 313L and an operation signalcorresponding to the operation for moving the threshold indicator 313Hor 313L is not supplied from the keyboard 2 or the mouse 3 to the GUIcontrol unit 211. In this case, the GUI control unit 211 supplies athreshold corresponding to a position (in the vertical direction) of thethreshold indicator 313H or 313L where the movement is finished to theFy-file managing unit 214 via the data-supply control unit 213 (FIG.43). The processing proceeds to step S260.

The threshold supplied to the Fy-file managing unit 214 via thedata-supply control unit 213 by the GUI control unit 211 is supplied tothe threshold processing unit 242 constituting the display-typedetermining unit 233 of the Fy-file managing unit 214 (FIG. 44).Thereafter, the threshold processing unit 242 performs processing usingthe threshold supplied from the GUI control unit 211 via the data-supplycontrol unit 213 until a new threshold is supplied to the thresholdprocessing unit 242 or a new Fy file is stored in the file storing unit231.

In step S260, the display-type determining unit 233 of the Fy-filemanaging unit 214 (FIG. 44) determines a display type anew (again) usingthe threshold supplied from the GUI control unit 211 to the thresholdprocessing unit 242 via the data-supply control unit 213 immediatelybefore the determination, i.e., the threshold indicated by the thresholdindicator 313H or 313L serving as the threshold information after themovement, and the variation stored in the Fy file stored in the Fy-filestoring unit 231.

The display-type determining unit 233 compares the variation stored inthe Fy file stored in the Fy-file storing unit 231 and the threshold Hor L indicated by the threshold indicator 313H or 313L serving as thethreshold information after the movement. The display-type determiningunit 233 determines a display type again on the basis of a result of thecomparison. The display type determined by the display-type determiningunit 233 again is written in the file storing unit 231 by thedisplay-type writing unit 234 to overwrite the display type in the Fyfile stored in the file storing unit 231.

Therefore, thereafter, in steps S217 and S224 in FIG. 48 describedabove, a display type for the frame of interest is judged with referenceto the display type determined by the display-type determining unit 233again. The frame of interest is displayed according to the display type.

As described above, the user changes the threshold H or L in determininga display type by operating the threshold indicator 313H or 313L servingas the GUI. A display type is determined again, so to speak, on a realtime basis using the threshold H or L after the change. Thus, the usercan change the threshold H or L taking into account performance and thelike of the PC 1 and easily adjust a display type (a display method) indisplaying a frame (an image) in scrub to make it possible to performsmooth (optimum) scrub.

After the processing in step S260, the processing returns to step S255.Thereafter, the same processing is repeated.

When it is judged in step S255 that the keyboard 2 or the mouse 3 isoperated to finish the editing program, this means that the user hasoperated the keyboard 2 or the mouse 3 to finish the editing program andan operation signal corresponding to the operation is supplied from thekeyboard 2 or the mouse 3 to the GUI control unit 211. In this case, theGUI display processing ends.

In FIG. 49, the variation graph, the position indicator 312, and thethreshold indicators 313H and 313L are displayed in the Fy-file displaysection 311. However, it is possible to display type information, whichis a GUI of belt-like areas representing, in time series, a display typein the Fy file managed by the Fy-file managing unit 214 (FIG. 43), inthe Fy-file display section 311 instead of or together with thevariation graph, the position indicator 312, and the thresholdindicators 313H and 313L.

FIG. 51 is a diagram showing an example of display of the Fy-filedisplay section 311 in which the type information is displayed togetherwith the variation graph, the position indicator 312, and the thresholdindicators 313H and 313L.

In FIG. 51, the Fy-file display section 311 includes a graph displaysection 321 and a type-information display section 322.

The graph display section 321 is a rectangular area on the upper side ofthe Fy-file display section 311. In the graph display section 321, thevariation graph, the position indicator 312, and the thresholdindicators 313H and 313L explained in FIG. 49 are displayed.

The type-information display section 322 is a rectangular area on thelower side of the Fy-file display section 311. The type information isdisplayed in the type-information display section 322.

The type information is a GUI having a horizontally rectangular area.The horizontal direction (the left to right direction) thereof indicateselapse of time, i.e., a frame (a place of a frame from the top) asexplained about the Fy-file display section 311 in FIG. 49.

The type information represents display types of frames corresponding torespective positions in the horizontal direction. In FIG. 51, assumingthat there are the three types, namely, the still image type V1, thenormal type V2, and the high display rate/low resolution type V3explained in FIG. 20, a still image section in which frames of the stillimage type V1 continue is shown with slanted lines down to the left (upto the right), a normal section in which frames of the normal type V2continue is shown without a pattern, and a high display rate/lowresolution section in which frames of the high display rate/lowresolution type V3 continue is shown with lines in the horizontaldirection (horizontal lines).

Therefore, the user can grasp a display type for a frame of a movingimage to be subjected to scrub according to the type information.

Scales in the horizontal direction of the variation graph displayed inthe graph display section 321 and the type information displayed in thetype-information display section 322 are identical. Therefore, whenattention is paid to a position in the horizontal direction of theFy-file display section 311 as a position of interest, a value of avariation graph in the position of interest represents a variation(motion information) of a frame corresponding to the position ofinterest. A pattern put on the type information in the position ofinterest represents a display type for the frame corresponding to theposition of interest.

It is assumed that, in the moving image to be subjected to scrub, all ofa minimum value of the number of continuing frames having a variationsmaller than the threshold L indicated by the threshold indicator 313L,a minimum value of the number of continuing frames having a variationequal to or larger than the threshold H indicated by the thresholdindicator 313H, and a minimum value of the number of continuing frameshaving a variation equal to or larger than the threshold L and smallerthan the threshold H are equal to or larger than the minimum limitnumber of frames N. Then, the still image type V1, the normal type V2,and the high display rate/low resolution type V3 are determined as adisplay type for the frames having a variation smaller than thethreshold L, a display type for the frames having a variation equal toor larger than the threshold L and smaller than the threshold H, and adisplay type for the frames having a variation equal to or larger thanthe threshold H, respectively.

In this case, as shown in FIG. 51, a section in which a value of thevariation graph (a variation) is smaller than the threshold L indicatedby the threshold indicator 313L in the graph display section 321 and asection of type information with the slanted lines, which is the stillimage section, in the type-information display section 322 coincide witheach other. A section in which a value of the variation graph is equalto or larger than the threshold H indicated by the threshold indicator313H in the graph display section 321 and a section of type informationwith the horizontal lines, which is the high display rate/low resolutionsection, in the type-information display section 322 coincide with eachother as well. A section in which a value of the variation graph isequal to or larger than the threshold L and smaller than the threshold Hand a section of type information without a pattern, which is the normalsection, in the type-information display section 322 in the graphdisplay section 321 coincide with each other as well.

Moreover, in the type-information display section 322, the positionindicator 312 displayed in the graph display section 321 is displayed asan extension of a segment as the position indicator 312 together withthe type information.

Therefore, the user can easily grasp (at a glance) a frame of interestdisplayed in (the display section 52 of) the viewer 51 (FIG. 49), i.e.,and variation and (a display method presented by) a display type for aframe corresponding to a position of the position indicator 312.

In the rectangular area as the type information, other than applying adifferent pattern to each of the display types, for example, it ispossible to apply a different color to each of the display type or applya different shade to each of the display types.

As shown in FIG. 51, when the Fy-file display section 311 includes thegraph display section 321 in which the variation graph and the like aredisplayed and the type-information display section 322 in which the typeinformation is displayed, as in the Fy-file display section 311 in FIG.49, it is possible to move the threshold indicators 313H and 313Laccording to operation by the user.

When the threshold indicator 313H or 313L is moved and, consequently,the threshold H or L used for determining a display type is changed, thedisplay-type determining unit 233 of the file managing unit 214 (FIG.44) compares the variation stored in the Fy file and the threshold H orL after the change and determines a display type again on the basis of aresult of the comparison. However, the type information of thetype-information display section 322 in FIG. 51 is displayed again inaccordance with the display type determined again.

FIG. 52 is a diagram showing type information displayed again as thethreshold H or L used for determining a display type is changed.

A diagram at the top in FIG. 52 shows the Fy-file display section 311displayed at certain timing. In FIG. 52, other than the Fy-file displaysection 311, a cursor operated by the keyboard 2 or the mouse 3 is alsoshown.

When the Fy-file display section 311 at the top in FIG. 52 is shown, forexample, the user drags to move the threshold indicator 313L in thedownward direction using the cursor. In this case, the display-typedetermining unit 233 of the file managing unit 214 (FIG. 44) comparesthe variation stored in the Fy file and the threshold L after the changeindicated by the threshold indicator 313L after the movement or thethreshold H not changed and determines a display type again on the basisof a result of the comparison. The type information is displayed againin accordance with the display type determined again by the display-typedetermining unit 233.

A second diagram from the top in FIG. 52 shows the Fy-file displaysection 311 having the type information displayed again as describedabove.

When the threshold indicator 313L is moved in the downward direction asdescribed above, the threshold (the threshold after the change) Lindicated by the threshold indicator 313L after the movement becomessmaller than the threshold (the threshold before the change) L indicatedby the threshold indicator 313L before the movement. Therefore, afterthe movement of the threshold indicator 313L, frames having a variationsmaller than the threshold L decrease and frames having a variationequal to or larger than the threshold L and smaller than the threshold Hincreases compared with those before the movement. In other words, afterthe movement of the threshold indicator 313L, frames, a display type forwhich is the still image type V1, decreases and frames, a display typefor which is the normal type V2, increases compared with those beforethe movement.

As a result, after the movement of the threshold indicator 313L, asshown in the second diagram from the top in FIG. 52, type information inwhich still image sections (portions with slanted lines) decrease andnormal sections (portions without slanted lines) increase compared withthose before the movement is displayed.

On the other hand, when the first Fy-file display section 311 from thetop in FIG. 52 is displayed, for example, the user drags to move thethreshold indicator 313L in the upward direction using the cursor. Inthis case, the display-type determining unit 233 of the file managingunit 214 (FIG. 44) compares the variation stored in the Fy file and thethreshold L after the change indicated by the threshold indicator 313Lafter the movement or the threshold H not changed and determines adisplay type again on the basis of a result of the comparison. The typeinformation is displayed again in accordance with the display typedetermined again by the display-type determining unit 233.

A third diagram from the top (at the bottom) in FIG. 52 shows theFy-file display section 311 having the type information displayed againas described above.

When the threshold indicator 313L is moved in the upward direction asdescribed above, the threshold (the threshold after the change) Lindicated by the threshold indicator 313L after the movement becomeslarger than the threshold (the threshold before the change) L indicatedby the threshold indicator 313L before the movement. Therefore, afterthe movement of the threshold indicator 313L, frames having a variationsmaller than the threshold L increase and frames having a variationequal to or larger than the threshold L and smaller than the threshold Hdecreases compared with those before the movement. In other words, afterthe movement of the threshold indicator 313L, frames, a display type forwhich is the still image type V1, increases and frames, a display typefor which is the normal type V2, decreases compared with those beforethe movement.

As a result, after the movement of the threshold indicator 313L, asshown in the third diagram from the top in FIG. 52, type information inwhich still image sections (portions with slanted lines) increase andnormal sections (portions without slanted lines) decrease compared withthose before the movement is displayed.

The GUI display processing at the time when type information isdisplayed as explained with reference to FIGS. 51 and 52 will beexplained with reference to a flowchart in FIG. 53.

In the GUI display processing, the GUI control unit 211 controls thedisplay control unit 218 to display the edition window 301 (FIG. 49).Consequently, the display control unit 218 causes the display 41 todisplay the edition window 301.

In step S281, as in step S251 in FIG. 50, the display control unit 218reads out the Fy file managed by the Fy-file managing unit 214 (FIG.43). The display control unit 218 causes the graph display section 321of the Fy-file display section 311 (FIG. 51) of the edition window 301to display a variation graph on the basis of a variation in the Fy file.The processing proceeds to step S282.

In step S282, the display control unit 218 causes the type-informationdisplay section 322 of the Fy-file display section 311 (FIG. 51) of theedition window 301 to display, on the basis of display types in the Fyfile managed by the Fy-file managing unit 214 (FIG. 43), typeinformation representing the display types in time series (typeinformation representing an arrangement of the display types in timeseries). The processing proceeds to step S283.

In step S283, as in step S252 in FIG. 50, the display control unit 218causes the graph display section 321 of the Fy-file display section 311(FIG. 51) of the edition window 301 to display the threshold indicators313H and 313L (FIG. 49) according to the thresholds (the defaultthresholds) used for determining the display type stored in the Fy filestored in the file storing unit 231 (FIG. 44) of the Fy-file managingunit 214.

Thereafter, the processing proceeds from step S283 to step S284. Insteps S284 to S291, the processing same as the processing in steps S253to S260 in FIG. 50 is performed.

In step S291, as in step S260 in FIG. 50, the display-type determiningunit 233 (FIG. 44) of the Fy-file managing unit 241 determines displaytypes again using the threshold indicated by the threshold indicator313H or 313L after the movement and the variation stored in the Fy filestored in the Fy-file storing unit 231 (FIG. 44). When the display typesdetermined again are written in the file storing unit 231 anew, theprocessing proceeds to step S292. The display control unit 218 (FIG. 43)performs update of display in the type-information display section 322for causing the type-information display section 322 of the Fy-filedisplay section 311 (FIG. 51) to display, on the basis of the displaytypes written in the file storing unit 231 of the Fy-file managing unit214 (FIG. 44) anew, type information representing the display types intime series (type information representing an arrangement of the displaytypes in time series) anew. The processing returns to step S286.

When the user operates the threshold indicator 313H or 313L serving asthe GUI as described above, the threshold H or L in determining displaytypes is changed. When display types are determined again using thethreshold H or L after the change, display of the type informationrepresenting the display types in time series is updated on the basis ofthe display types determined again.

As the variation, there are motion information and fineness information.It is possible to store only one of the motion information and thefineness information in the Fy file. It is also possible to store boththe motion information and the fineness information in the Fy file asshown in, for example, FIG. 35.

When both the motion information and the fineness information are storedin the Fy file, in the Fy-file display section 311, for example, it ispossible to select one of the motion information and the finenessinformation serving as the variations in the Fy file and display avariation graph representing the variation selected in time series.

FIG. 54 is a diagram showing (the edition window 301 including) theFy-file display section 311 for selecting one of the variations, i.e.,the motion information and the fineness information, and displaying avariation graph representing the variation selected in time series.

In FIG. 54, a motion information tab 331 and a fineness information tab332 serving as GUIs operated in selecting which of a variation graphrepresenting the motion information in time series and a variation graphrepresenting the fineness information in time series is selected isprovided in the upper left of the Fy-file display section 311.

When the user performs operation for selecting, for example, the motioninformation tab 331 of the motion information tab 331 and the finenessinformation tab 332 (e.g., clicks the motion information tab 331 withthe mouse 3), a graph representing the motion information in time seriesis displayed in the Fy-file display section 311 as a variation graph.When the user performs operation for selecting, for example, thefineness information tab 332 of the motion information tab 331 and thefineness information tab 332, a graph representing the finenessinformation in time series is displayed in the Fy-file display section311 as a variation graph.

In FIG. 54, operation for selecting the fineness information tab 332 isperformed, whereby the graph representing the fineness information intime series is displayed in the Fy-file display section 311 as avariation graph.

When both the motion information and the fineness information are storedin the Fy file, it is possible to display a variation graph of themotion information or the fineness information in the Fy-file displaysection 311. Moreover, it is possible to display variation graphs ofboth the motion information and the fineness information. It is alsopossible to display type information representing the display types intimes series together with the variation graph.

FIG. 55 is a diagram showing the Fy-file display section 311.

The Fy-file display section 311 in FIG. 55 includes the graph displaysection 321 in which a variation graph and the like are displayed andthe type-information display section 322 in which type information andthe like are displayed.

On the upper left of the Fy-file display section 311 in FIG. 55, themotion information tab 331, the fineness information tab 332, and an allinformation tab 333 serving as GUIs are provided. These tabs areoperated in selecting which of a variation graph representing motioninformation in time series (hereinafter also referred to as motioninformation graph as appropriate), a variation graph representingfineness information in time series (hereinafter also referred to asfineness information graph as appropriate), or both the motioninformation graph and the fineness information graph is displayed.

When the user performs, for example, operation for selecting the motioninformation tab 331 among the motion information tab 331, the finenessinformation tab 332, and the all information tab 333, as shown in theupper part in FIG. 55, the motion information graph is displayed in thegraph display section 321 of the Fy-file display section 311. In thetype-information display section 322 of the Fy-file display section 311,type information representing display types (display types based on themotion information), which is determined by comparing the motioninformation and the thresholds H and L, in time series is displayed.Besides, in the Fy-file display section 311, the position indicator 312indicating a position of a frame (a frame of interest) displayed in theviewer 51 (FIG. 49) and the threshold indicators 313H and 313Lindicating the thresholds H and L used in determining display typesbased on the motion information, respectively, are also displayed.

When the motion information and the fineness information are stored inthe Fy file, for example, as explained with reference to FIGS. 38 to 40,the motion information is compared with the two thresholds H and L andthe fineness information is compared with the one threshold K.Consequently, the still image type VC1, the normal type VC2, the highdisplay rate/low resolution type VC3, or the normal display rate/lowresolution type VC4 shown in FIG. 38 is determined as a display type (afinal display type) based on the motion information and the finenessinformation.

In this case, when the user performs, for example, operation forselecting the all information tab 333 among the motion information tab331, the fineness information tab 332, and the all information tab 333,as shown in the lower part in FIG. 55, the motion information graph andthe fineness information graph are displayed in the graph displaysection 321 of the Fy-file display section 311. In the type-informationdisplay section 322 of the Fy-file display section 311, type informationrepresenting display types based on the motion information and thefineness information, which are determined by comparing the motioninformation and the thresholds H and L and comparing the finenessinformation and the threshold K, in time series is displayed. Besides,in the Fy-file display section 311, the position indicator 312indicating a position of a frame (a frame of interest) displayed in theviewer 51 (FIG. 49) and the threshold indicators 313H, 313L, and 313Kindicating the thresholds H, L, and K used in determining display types,respectively, are also displayed.

When the user performs, for example, operation for selecting thefineness information tab 332 among the motion information tab 331, thefineness information tab 332, and the all information tab 333, as in thecase in which the operation for selecting the motion information tab 331is performed, the fineness information graph is displayed in the graphdisplay section 321 of the Fy-file display section 311. In thetype-information display section 322 of the Fy-file display section 311,type information representing display types (display types based on thefineness information), which are determined by comparing the finenessinformation and the threshold K, in time series is displayed. Besides,in the Fy-file display section 311, the position indicator 312indicating a position of a frame (a frame of interest) displayed in theviewer 51 (FIG. 49) and the threshold indicator 313K indicating thethreshold K used in determining display types are also displayed.

The GUI display processing performed when plural kinds of information(values) such as the motion information and the fineness information arestored in the Fy file as variations as explained with reference to FIGS.54 and 55 will be explained with reference to a flowchart in FIG. 56.

The motion information and the fineness information are stored in the Fyfile as the variations. For example, motion information of the motioninformation and the fineness information is set as default ofinformation of interest that is a variation of interest. Moreover, whendisplay types based on the motion information, which is the defaultinformation of interest, is not stored in the Fy file, in step S214 ofthe processing of scrub in FIG. 48, the Fy managing unit 214 (FIG. 43)determines display types based on the motion information and writes thedisplay types in the file storing unit 231 (FIG. 44).

In the GUI display processing, the GUI control unit 211 (FIG. 43)controls the display control unit 218 (FIG. 43) to display the editionwindow 301 (FIG. 49). Consequently, the display control unit 218 causesthe display 41 to display the edition window 301. In step S311, thedisplay control unit 218 reads out the Fy file managed by the Fy-filemanaging unit 214 (FIG. 43), i.e., the Fy file stored in the filestoring unit 231 (FIG. 44) of the Fy-file managing unit 214. The displaycontrol unit 218 causes the Fy-file display section 311 of the editionwindow 301 to display a variation graph (a motion information graph) onthe basis of the motion information, which is the default information ofinterest, of the motion information and the fineness information storedin the Fy file.

Moreover, the display control unit 218 causes the Fy-file displaysection 311 to display the motion information tab 331, the finenessinformation tab 332, and the all information tab 333, for example, asshown in FIG. 55, on the basis of the motion information and thefineness information stored in the Fy file read out from the filestoring unit 231 of the Fy-file managing unit 214. The processingproceeds from step S311 to step S312.

In step S312, the display control unit 218 causes the type-informationdisplay section 322 of the Fy-file display section 311 (FIG. 51) of theedition window 301 to display, on the basis of the display types basedon the motion information, which is the default information of interest,stored in the file storing unit 231 of the Fy-file managing unit 214(FIG. 43), type information representing the display types in timeseries (type information representing an arrangement of the displaytypes in time series). The processing proceeds to step S313.

In step S313, the display control unit 218 causes the Fy-file displaysection 311 to display the threshold indicators 313H and 313L (FIG. 55)according to the thresholds H and L used for determining the displaytypes based on the motion information, which is the default informationof interest, stored in the Fy file stored in the file storing unit 231(FIG. 44) of the Fy-file managing unit 214.

Thereafter, the processing proceeds from step S313 to S314. In stepsS314 to S317, processing same as steps S253 to S256 in FIG. 50 isperformed.

In step S314, the display control unit 218 acquires information on aposition of the scrub dial 54 and a frame allocated to a designatedsection (FIG. 45) in which the scrub dial 54 is located, i.e., a frameof interest as well from the GUI control unit 211 (receives theinformation on the frame of interest supplied from the GUI control unit211). The processing proceeds to step S315.

In step S315, the display control unit 218 causes the Fy-file displaysection 311 to display the position indicator 312 (FIG. 55) according tothe information on the frame of interest acquired from the GUI controlunit 211 in the immediately preceding step S314. The display controlunit 218 causes the Fy-file display section 311 to display the positionindicator 312 in a position corresponding to the frame (the frame ofinterest) allocated to the designated section in which the scrub dial 54is located.

The processing proceeds from step S315 to step S316. The GUI controlunit 211 judges whether the keyboard 2 or the mouse 3 is operated tofinish the editing program.

When it is judged in step S316 that the keyboard 2 or the mouse 3 is notoperated to finish the editing program, the processing proceeds to stepS317. The GUI control unit 211 judges whether the scrub dial 54 ismoved, i.e., the user has moved the scrub dial 54 by operating thekeyboard 2 or the mouse 3.

When it is judged in step S317 that the scrub dial 54 is moved, thismeans that the user has operated the keyboard 2 or the mouse 3 to movethe scrub dial 54 and an operation signal corresponding to the operationis supplied from the keyboard 2 or the mouse 3 to the GUI control unit211. In this case, the GUI control unit 211 sets a frame allocated to adesignated position (FIG. 45) in which the scrub dial 54 is located as aframe of interest and supplies information on the frame of interest tothe display control unit 218. The processing returns to step S314.Thereafter, the same processing is repeated.

On the other hand, when it is judged in step S317 that the scrub dial 54is not moved, the processing proceeds to step S318. The GUI control unit211 (FIG. 43) judges whether operation for selecting the motioninformation tab 331, the fineness information tab 332, or the allinformation tab 333 displayed in step S311 is performed.

When it is judged in step S318 that the operation for selecting themotion information tab 331, the fineness information tab 332, or the allinformation tab 333 is not performed, steps S319 to S322 described laterin which processing for changing the information of interest isperformed are skipped. The processing proceeds to step S323.

When it is judged in step S318 that the operation for selecting themotion information tab 331, the fineness information tab 332, or the allinformation tab 333 is performed, this means that the user has operatedthe keyboard 2 or the mouse 3 to select the motion information tab 331,the fineness information tab 332, or the all information tab 333 and anoperation signal corresponding to the selection is supplied from thekeyboard 2 or the mouse 3 to the GUI control unit 211 (FIG. 43). In thiscase, the GUI control unit 211 sets information corresponding to theselected one of the motion information tab 331, the fineness informationtab 332, and the all information tab 333 (hereinafter also referred toas selected tab as appropriate) as information of interest on the basisof the operation signal from the keyboard 2 or the mouse 3. Theprocessing proceeds to step S319.

When the selected tab is the motion information tab 331, the motioninformation that is information corresponding to the motion informationtab 331 is set as information of interest. When the selected tab is thefineness information tab 332, the fineness information corresponding tothe fineness information tab 332 is set as information of interest. Whenthe selected tab is the all information tab 333, all the variationsstored in the Fy file, i.e., both the motion information and thefineness information corresponding to the all information tab 333 areset as information of interest.

In step S319, the display control unit 218 (FIG. 43) reads out the Fyfile managed by the Fy-file managing unit 214, i.e., the Fy file storedin the file storing unit 231 (FIG. 44) of the Fy-file managing unit 214.The display control unit 218 causes the Fy-file display section 311 ofthe edition window 301 to display the variation graph on the basis ofthe information of interest of the motion information and the finenessinformation stored in the Fy file.

For example, when the motion information of the motion information andthe fineness information is the information of interest, in step S319,the display control unit 218 causes the Fy-file display section 311 todisplay the motion information graph as shown in the upper part in FIG.55. For example, when the fineness information is the information ofinterest, in step S319, the display control unit 218 causes the Fy-filedisplay section 311 to display the fineness information graph. Moreover,for example, when both the motion information and the finenessinformation are the information of interest, in step S319, the displaycontrol unit 218 causes the Fy-file display section 311 to display themotion information graph and the fineness information graph as shown inthe lower part in FIG. 55.

Thereafter, the processing proceeds from step S319 to step S320. TheFy-file managing unit 214 (FIG. 43) compares, in the display-typedetermining unit 233, the information of interest and a presentthreshold for determining display types based on the information ofinterest (the default threshold or the threshold H, L, or K representedby a position of the threshold indicator 313H, 313L, or 313K (FIG. 55))to determine display types based on the information of interest andcauses the file storing unit 231 to store the display types. Theprocessing proceeds to step S321.

In step S321, the display control unit 218 causes the Fy-file displaysection 311 to display, on the basis of the display types based on theinformation of interest stored in the file storing unit 231 of theFy-file managing unit 214 (FIG. 43), type information representing thedisplay types in time series (type information representing anarrangement of the display types in time series). The processingproceeds to step S322.

In step S322, the display control unit 218 causes the Fy-file displaysection 311 to display the threshold indicator 313H, 313L, or 313K (FIG.55) according to the threshold H, L, or K used for determining thedisplay types based on the information of interest stored in the filestoring unit 231 (FIG. 44) of the Fy-file managing unit 214 (used fordetermining the display types in the immediately preceding step S320).

For example, when the motion information of the motion information andthe fineness information is the information of interest, in step S322,the display control unit 218 causes the Fy-file display section 311 todisplay the threshold indicators 313H and 313L in positions thereoncorresponding to the thresholds H and L used for determining the displaytypes based on the motion information as shown in the upper part in FIG.55.

For example, when the fineness information is the information ofinterest, in step S322, the display control unit 218 causes the Fy-filedisplay section 311 to display the threshold indicator 313K in aposition thereon corresponding to the threshold K used for determiningthe display types based on the fineness information.

Moreover, for example, when both the motion information and the finenessinformation are the information of interest, in step S322, the displaycontrol unit 218 causes the Fy-file display section 311 to display thethreshold indicators 313H, 313L, and 313K in positions thereoncorresponding to the thresholds H, L, and K used for determining thedisplay types based on the motion information and the finenessinformation, respectively, as shown in the lower part in FIG. 55.

Thereafter, the processing proceeds from step S322 to step S323. Insteps S323 to S327, processing same as the processing in steps S288 toS292 in FIG. 53 is performed.

In step S323, the GUI control unit 211 judges whether movement of thethreshold indicator displayed in the Fy-file display section 311, i.e.,the threshold indicator corresponding to the threshold used fordetermining the display types based on the information of interest (oneof the threshold indicators 313H, 313L, and 313K) is started, i.e.,whether the user has operated the keyboard 2 or the mouse 3 to move thethreshold indicator.

When it is judged in step S323 that the movement of the thresholdindicator displayed in the Fy-file display section 311 is not started,the processing returns to step S316. Thereafter, the same processing isrepeated.

When it is judged in step S323 that the movement of the thresholdindicator displayed in the Fy-file display section 311 is started, thismeans that the user has operated the keyboard 2 or the mouse 3 to movethe threshold indicator displayed in the Fy-file display section 311 andan operation signal corresponding to the operation is supplied from thekeyboard 2 or the mouse 3 to the GUI control unit 211. In this case, theGUI control unit 211 calculates, according to the operation signal fromthe keyboard 2 or the mouse 3, a position to which the thresholdindicator displayed in the Fy-file display section 311 is moved. The GUIcontrol unit 211 supplies information representing the position to thedisplay control unit 218. The processing proceeds to step S324.

In step S324, the display control unit 218 causes the Fy-file displaysection 311 to display the threshold indicator in a positioncorresponding to the information from the GUI control unit 211 insteadof the position where the threshold indicator was displayed immediatelybefore the step. The processing proceeds to step S325.

In step S325, the GUI control unit 211 judges whether the movement ofthe threshold indicator is finished, i.e., whether the user has finishedthe operation of the keyboard 2 or the mouse 3 for moving the thresholdindicator.

When it is judged in step S325 that the operation for moving thethreshold indicator is not finished, this means that the user continuesto operate the keyboard 2 or the mouse 3 to move the threshold indicatorand an operation signal corresponding to the operation is supplied fromthe keyboard 2 or the mouse 3 to the GUI control unit 211. In this case,the GUI control unit 211 calculates, according to the operation signalfrom the keyboard 2 or the mouse 3, a position to which the thresholdindicator is moved and supplies information representing the position tothe display control unit 218. The processing returns to step S324.Thereafter, the same processing is repeated.

On the other hand, when it is judged in step S325 that the operation formoving the threshold indicator is finished, this means that the user hasfinished the operation of the keyboard 2 or the mouse 3 for moving thethreshold indicator and an operation signal corresponding to theoperation for moving the threshold indicator is not supplied from thekeyboard 2 or the mouse 3 to the GUI control unit 211. In this case, theGUI control unit 211 supplies a threshold corresponding to a position(in the vertical direction) in which the movement of the thresholdindicator is finished to the Fy-file managing unit 214 via thedata-supply control unit 213 (FIG. 43). The processing proceeds to stepS326.

As explained with reference to FIG. 50, the threshold supplied to theFy-file managing unit 214 via the data-supply control unit 213 by theGUI control unit 211 is supplied to the threshold processing unit 242constituting the display-type determining unit 233 of the Fy-filemanaging unit 214 (FIG. 44). Thereafter, the threshold processing unit242 performs the processing using the threshold supplied from the GUIcontrol unit 211 via the data-supply control unit 213 until a newthreshold is supplied to the threshold processing unit 242 or a new Fyfile is stored in the file storing unit 231.

In step S326, the display-type determining unit 233 of the Fy-filemanaging unit 214 (FIG. 44) determines display types anew (again) usingthe threshold supplied from the GUI control unit 211 to the thresholdprocessing unit 242 via the data-supply control unit 213 immediatelybefore the step, i.e., the threshold indicated by the thresholdindicator after the movement and the variation, which is the informationof interest, stored in the Fy file stored in the Fy-file storing unit231.

The display-type determining unit 233 compares the variation, which isthe information of interest, stored in the Fy file stored in the Fy-filestoring unit 231 and the threshold indicated by the threshold indicatorafter the movement. The display-type determining unit 233 determinesdisplay types based on the information of interest again on the basis ofa result of the comparison. The display types based on the informationof interest determined again by the display-type determining unit 233 iswritten in the file storing unit 231 to overwrite the display typesstored therein.

After the display types based on the information of interest aredetermined in step S326 as described above, the processing proceeds tostep S327. The display control unit 218 (FIG. 43) performs update ofdisplay in the type-information display section 322 for causing theFy-file display section 311 to display type information representing thedisplay types in time series (type information representing anarrangement in time series of the display types) on the basis of thedisplay types based on the information of interest written in the filestoring unit 231 of the Fy-file managing unit 214 (FIG. 44) anew. Theprocessing returns to step S316.

When it is judged in step S316 that the keyboard 2 or the mouse 3 isoperated to finish the editing program, this means that the user hasoperated the keyboard 2 or the mouse 3 to finish the editing program andan operation signal corresponding to the operation is supplied from thekeyboard 2 or the mouse 3 to the GUI control unit 211. In this case, theGUI display processing ends.

In scrub, a position of the scrub dial 54 is detected at a periodcorresponding to a display rate. A frame allocated to a designatedsection in which the scrub dial 54 is located is set as a frame ofinterest. The frame of interest is displayed in the viewer 51 (FIG. 3).

For simplification of the explanation, it is assumed that a display rateis fixed.

As explained with reference to FIG. 45, a movable range of the scrub bar53 is divided into designated sections that are provided in the samenumber as the number of frames forming a moving image to be subjected toscrub and have an identical section length. In this case, when the scrubdial 54 is moved at fixed speed, the scrub dial 54 is moved a fixeddistance and in a fixed number of designated sections in a unit time.Therefore, as described above, if a display rate is fixed, when thescrub dial 54 is moved at the fixed speed by the fixed distance, a fixednumber of frames are displayed. This does not depend on in what kind ofrange in the movable range the scrub bar 53 is moved.

For example, in the moving image to be subjected to scrub, there are asection in which frames having intense motion continue (hereinafterreferred to as section with motion as appropriate) and a section inwhich frames not having motion (hereinafter referred to as sectionwithout motion as appropriate) In this case, even if the scrub dial 54is moved at the fixed speed by the fixed distance in a range ofdesignated section to which the frames in the section with motion areallocated and a range of designated sections to which the frames in thesection without motion are allocated among the movable range of thescrub bar 53, the number of frames displayed in the viewer 51 is thesame.

However, when the scrub dial 54 is moved in the range of the designatedsections to which the frames in the section with motion are allocated,an image displayed in the viewer 51 substantially changes (moves)because the frames in the section with motion have intense motion.

On the other hand, when the scrub dial 54 is moved in the range of thedesignated section to which the frames in the section without motion areallocated, an image displayed in the viewer 51 does not change (changeslittle) because the frames in the section without motion do not havemotion (have little motion).

Therefore, in the case in which the movable range of the scrub bar 53 isdivided into the designated sections having the identical sectionlength, when the scrub dial 54 is moved at the fixed speed by the fixeddistance, an image displayed in the viewer 51 substantially changes insome ranges and does not change in other ranges.

In the range of the designated sections to which the frames in thesection without motion are allocated, even if the scrub dial 54 is movedby a certain degree, an image displayed in the viewer 51 does notchange. Thus, the user operating the scrub dial 54 sometimes feels itannoying.

On the other hand, in the range of the designated sections to which theframes in the section with motion are allocated, when the scrub dial 54is moved by a certain degree, an image displayed on the viewer 51substantially changes. Thus, the user operating the scrub dial 54sometimes needs to perform precise (careful) operation of the scrub dial54 in order to find a frame of a desired image.

Thus, a section length of designated sections to which frames areallocated is not fixed but is weighted according to, for example, motioninformation as variation in the frames allocated to the designatedsections. This makes it possible to divide the movable range of thescrub bar 53 into designated sections that have a larger section lengthwhen frames having larger motion information as variables are allocatedto the designated sections.

FIG. 57 is a diagram showing an example of weight in weighting thesection length of the designated sections.

In FIG. 57, the abscissa indicates a frame (a place of a frame from thetop) and the ordinate indicates motion information serving as avariation.

In FIG. 57, thresholds L₁, L₂, L₃, and L₄ satisfying a relationL₁>L₂>L₃>L₄ are used. A weight w for a section length of designatedsections to which frames having motion information equal to or largerthan the threshold L₁ are allocated is set as, for example, 1. A weightw for a section length of designated sections to which frames havingmotion information equal to or larger than the threshold L₂ and smallerthan the threshold L₁ are allocated is set as, for example, 0.7. Aweight w for a section length of designated sections to which frameshaving motion information equal to or larger than the threshold L₃ andsmaller than the threshold L₂ are allocated is set as, for example, 0.5.A weight w for a section length of designated section to which frameshaving motion information equal to or larger than the threshold L₄ andsmaller than the threshold L₃ are allocated is set as, for example, 0.3.Moreover, a weight w for a section length of designated sections towhich frames having motion information smaller than the threshold L₄ areallocated is set as, for example, 0.1.

It is possible to calculate the section length weighted using theweights w, for example, as explained below.

A section in which designated sections having a section length weightedby the identical weight w is referred to as an identical weight section.A sequence of designated sections allocated to frames forming a movingimage to be subjected to scrub is divided into Q identical weightsections.

The number of designated sections forming a qth identical weight sectionfrom the top among the Q identical weight sections and the number offrames allocated to the designated sections forming the qth identicalweight section are represented as F_(q). A weight given to a sectionlength of each of the designated sections forming the qth identicalweight section is represented as w_(q).

A total number F of the frames of the moving image to be subjected toscrub is represented as F=F₁+F₂+ . . . +F_(Q).

Assuming that length of the movable range is 1, a section length ofdesignated sections is represented as 1/F when the section length is notweighted.

On the other hand, a section length of designated sections forming theqth identical weight section from the top is represented asw_(q)/(Σw_(i)F_(i)) when the section length is weighted. Since length ofthe qth identical weight section from the top is represented asw_(q)F_(q)/(Σw_(i)F_(i)), by dividing w_(q)F_(q)/(Σw_(i)F_(i)) by thenumber Fq of (the frames allocated to) the designated sections formingthe qth identical weight section, it is possible to calculate a sectionlength of the designated sections forming the qth identical weightsection from the top at the time when the section length is weighted.Σw_(i)F_(i) represents a sum of w_(i)F_(i) when a suffix i is changedfrom 1 to Q.

It is possible to calculate a section length of designated sections atthe time when the section length is not weighted by setting all theweights w₁, w₂, . . . , w_(Q) at the time when the section length isweighted to the identical weight w. The section length of the designatedsections forming the qth identical weight section from the top at thetime when the section length is weighted is w_(q)/(Σw_(i)F_(i)) asdescribed above. When w₁, w₂, . . . , and w_(Q) are set as the identicalweight w, w_(q)/(Σw_(i)F_(i))=w/(wΣF_(i))=1/F_(i)=1/F.

In FIG. 57, the weight w for the section length of the designatedsections allocated to the frames is calculated using the four thresholdsL₁, L₂, L₃, and L₄ that are not related to the thresholds used fordetermining display types. However, besides, it is also possible tocalculate a weight w for section length using the thresholds used fordetermining display types.

FIG. 58 is a diagram showing an example of weights calculated using thethresholds H and L used for determining display types based on motioninformation.

In FIG. 58, the abscissa indicates a frame (a place of a frame from thetop) and the ordinate indicates motion information serving as avariation.

In FIG. 58, a weight w for a section length of designated sections towhich frames having motion information equal to or larger than thethreshold H are allocated is set as, for example, 0.5. A weight w for asection length of designated sections to which frames having motioninformation equal to or larger than the threshold L and smaller than thethreshold H are allocated is set as, for example, 0.3. A weight w for asection length of designated sections to which frames having motioninformation smaller than the threshold L are allocated is set as, forexample, 0.1.

Besides, for example, when display types of frames are determined on thebasis of motion information of the frames, it is possible to determine aweight w for a section length of designated sections allocated to theframes according to the display types.

When display types are determined on the basis of motion information,the display types represents degrees of motion of frames. Specifically,for example, as shown in FIG. 20, when the still image type V1, thenormal type V2, or the high display rate/low resolution type V3 isdetermined as a display on the basis of motion information, inprinciple, frames of the still image type V1 are frames having motioninformation smaller than the threshold L and frames of the normal typeV2 are frames having motion information equal to or larger than thethreshold L and smaller than the threshold H. Frames of the high displayrate/low resolution type V3 are frames having motion information equalto or larger than the threshold H.

Therefore, as in the case in which the thresholds H and L are usedexplained with reference to FIG. 58, it is possible to set a weight wfor a section length of designated sections to which the frames of thestill image type V1 are allocated, a weight w for section length ofdesignated sections to which the frames of the normal type V2 areallocated, and a weight w for section length of designated sections towhich the frames of the high display rate/low resolution type V3 areallocated as 0.1, 0.3, and 0.5, respectively.

When a weight w for section length of designated sections allocated toframes is determined according to display types of the frames, all of astill image section in which the frames of the still image type V1continue, a normal section in which the frames of the normal type V2continue, and a high display rate/low resolution section in which theframes of the high display rate/low resolution type V3 continue areidentical weight sections.

FIG. 59 is a diagram showing designated sections at the time when asection length is weighted and at the time when a section length is notweighted, respectively.

When the section length is not weighted, as shown in the upper half inFIG. 59, the movable range of the scrub bar 53 is divided intodesignated sections that are provided in the same number as the numberof frames forming a moving image to be subjected to scrub and have anidentical section length. As explained with reference to FIG. 45, theframes forming the moving image to be subjected to scrub are allocatedin time series from a designated section on the left.

When the section length is not weighted, a section length of designatedsections allocated to frames in all the still image sections in whichthe frames of the still image type V1 continue, the normal section inwhich the frames of the normal type V2 continue, and the high displayrate/low resolution section in which the frames of the high displayrate/low resolution type V3 continue are d.

On the other hand, when the section length is weighted, as shown in thelower half in FIG. 59, the movable range of the scrub bar 53 is dividedinto designated sections. As explained with reference to FIG. 45, theframes forming the moving image to be subjected to scrub are allocatedin time series from a designated section on the left. However, a sectionlength is larger in a designated section to which a frame having largermotion information serving as a variation is allocated.

In FIG. 59, a section length of designated sections allocated to framesin the still image section in which the frames of the still image typeV1 continue, i.e., frames without motion is d/2, which is ½ time aslarge as the section length d at the time when the section length is notweighted. A section length of designated sections allocated to frames inthe normal section in which the frames of the normal type V2 continue,i.e., frames with motion, although not intense, is d, which is identicalwith the section length d at the time when the section length is notweighted. Moreover, a section length of designated sections allocated toframes in the high display rate/low resolution section in which theframes of the high display rate/low resolution type V3 continue, i.e.,frames with intense motion is 3d, which is three times as large as thesection length d at the time when the section length is not weighted.

Therefore, in weighting the section length, when the scrub dial 54 ismoved by an amount of movement ½ time as large as that at the time whenthe section length is not weighted, display of the frames in the stillimage section ends. As a result, even if the scrub dial 54 is moved by acertain degree, since an image displayed in the viewer 51 does notchange, it is possible to prevent the user operating the scrub dial 54from feeling the movement of the scrub dial 54 annoying.

In the case in which the section length is weighted, when the scrub dial54 is moved by an amount of movement three times as large as that at thetime when the section length is not weighted, a frame of interestdisplayed in the viewer 51 is changed from a frame in the high displayrate/low resolution section to the next frame. As a result, the user caneasily find a frame of a desired image without operating the scrub dial54 so precisely (carefully).

In FIG. 59, type information in which a scale in the horizontaldirection is identical with a scale in the movable range of the scrubbar 53 (a GUI of belt-like areas representing display types in timeseries).

In type information in which a scale in the horizontal direction isidentical with a scale in the movable range of the scrub bar 53, (apattern of) type information in a position of the scrub dial 54represents a display type for a frame allocated to a designated sectionin which the scrub dial 54 is located.

In FIG. 59, among the belt-like areas serving as the type information,an area D1 with slanted lines represents a still image section in whichthe frames of the still image type V1 continue and an area D2 without apattern represents a normal section in which the frames of the normaltype V2 continue. An area D3 with horizontal lines represents a highdisplay rate/low resolution section in which the frames of the highdisplay rate/low resolution type V3 continue.

In FIG. 59, as describe above, a section length of designated sectionsallocated to frames in the still image section in which the frames ofthe still image type V1 continue at the time when the section length isweighted is d/2, which is ½ time as large as the section length d at thetime when the section length is not weighted. Thus, length in thehorizontal direction of the area D1 representing the still image sectionat the time when the section length is weighted is also ½ time as largeas that at the time when the section length is not weighted.

In FIG. 59, a section length of designated sections allocated to framesin the normal section in which the frames of the normal type V2 continueat the time when the section length is weighted is d, which is identicalwith the section length d at the time when the section length is notweighted. Thus, length in the horizontal direction of the area D2representing the normal section at the time when the section length isweighted is also identical with that at the time when the section lengthis not weighted.

Moreover, in FIG. 59, a section length of designated sections allocatedto frames in the high display rate/low resolution section in which theframes of the high display rate/low resolution type V3 continue at thetime when the section length is weighted is 3d, which is three times aslarge as the section length d at the time when the section length is notweighted. Thus, length in the horizontal direction of the area D3representing the high display rate/low resolution section at the timewhen the section length is weighted is also three times as large as thatat the time when the section length is not weighted.

As described above, when the section length is weighted, typeinformation depends on the section length after the weighting and isdifferent from that at the time when the section length is not weighted(in other words, when the section length is not weighted, typeinformation is different from that at the time when the section lengthis weighted).

In the editing system in FIG. 43, a weighting-on mode for weighting asection length of designated sections and a weighing-off mode for notweighting the section length are provided as operation modes. It ispossible to switch an operation mode to the weighting-on mode or theweighting-off mode according to operation by the user.

Processing applied to section length of designated sections by theediting system in FIG. 43 (processing concerning a section length) willbe explained with reference to a flowchart in FIG. 60.

A not-shown weight button that is operated in switching the operationmode to the weighting-on mode or the weighting-off mode is provided inthe edition window 301 (FIG. 49). When the user operates the weightbutton, the operation mode is switching to the weighting-on mode or theweighting-off mode.

It is assumed that length of the movable range of the scrub bar 53 is 1.

In the processing concerning a section length, in step S351, the GUIcontrol unit 211 judges whether the operation mode is switched. When itis judged in step S351 that the operation mode is switched to theweighting-on mode, the processing proceeds to step S352. The GUI controlunit 211 calculates a weight given to the weight length of each of thedesignated sections allocated to a frame as described above on the basisof the variation or the display type in the Fy file stored in the filestoring unit 231 of the Fy-file managing unit 214 (FIG. 44). Theprocessing proceeds to step S353.

In step S353, the GUI control unit 211 calculates a weighted sectionlength for each of the designated sections on the basis of the weightsgiven to the section length of each of the sections allocated to theframe.

When the weight given to the section length of each of the designatedsections allocated to the frame is calculated as described above on thebasis of the variation or the display type in step S352, the GUI controlunit 211 recognizes a section in which designated sections having asection length weighted by the identical weight w continue as anidentical weight section.

As described above, a sequence of designated section allocated to framesforming a moving image to be subjected to scrub is divided into Qidentical weight sections. Then, the GUI control unit 211 calculates asection length w_(q)/(Σw_(i)F_(i)) of designated sections forming a qthidentical weight section from the top using the number F_(q) ofdesignated sections forming the qth identical weight section from thetop among the Q identical weight sections and the weight w_(q) given tothe section length of the designated sections forming the qth identicalweight section calculated in step S353.

After calculating the section length w_(q)/(Σw_(i)F_(i)) of thedesignated sections forming each of the Q identical weight sections instep S353, the GUI control unit 211 supplies the section lengthw_(q)/(Σw_(i)F_(i)) of the designated sections to the display controlunit 218 (FIG. 43). The processing proceeds to step S354. The GUIcontrol unit 211 sets (changes) the section length of the designatedsections dividing the movable range of the scrub bar 53 displayed in theviewer 51 to the section length calculated in step S353. The processingproceeds to step S355.

For example, as shown in FIG. 51, when type information is displayed inthe Fy-file display section 311, in step S355, the display control unit218 changes the type information displayed in the Fy-file displaysection 311 as explained with reference to FIG. 59 according to thesection length w_(q)/(Σw_(i)F_(i)) of the designated sections suppliedfrom the GUI control unit 211 in step S353. The processing returns tostep S351.

On the other hand, when it is judged in step S351 that the operationmode is switched to the weighting-off mode, the processing proceeds tostep S356. The GUI control unit 211 sets (changes) the section lengthsof each of the designated sections allocated to the frame to anidentical length 1/F and supplies the section length 1/F to the displaycontrol unit 218. The processing proceeds to step S357. As describedabove, F represents a total number of frames forming a moving image tobe subjected to scrub.

For example, as shown in FIG. 51, when the type information is displayedin the Fy-file display section 311, in step S357, the display controlunit 218 changes the type information displayed in the Fy-file displaysection 311 according to the section length 1/F supplied from the GUIcontrol unit 211 in step S356. The processing returns to step S351.

In the Fy-file display section 311, display allocation range informationrepresenting an allocation range, which is a range of frames allocatedto the front designated section to the end designated section in themovable range of the scrub bar 53, is displayed. It is possible toallocate a front frame to an end frame in the allocation rangerepresented by the allocation range information to the front designatedsection to the end designated section in the movable range of the scrubbar 53.

FIG. 61 is a diagram showing an example of display of the Fy-filedisplay section 311 in which range designation indicators 351S and 351Eserving as allocation range information are displayed.

As explained with reference to FIG. 51, the Fy-file display section 311in FIG. 61 includes the graph display section 321 and thetype-information display section 322. The variation graph, the typeinformation, the position indicator 312, and the threshold indicators313H and 313L are displayed in the Fy-file display section 311.

Moreover, in the Fy-file display section 311 in FIG. 61, the rangedesignation indicators 351S and 351E representing an allocation range,which is a range of frames allocated to the front designated section tothe end designated section in the movable range of the scrub bar 53, aredisplayed.

A state in which the front designated section to the end designatedsection in the movable range of the scrub bar 53 are allocated to thefront frame to the end frame of the moving image to be subjected toscrub is referred to as a default allocation state. In the defaultallocation state, as explained with reference to FIG. 49, if length inthe horizontal direction of the area of the Fy-file display section 311in which the variation graph is displayed and length of the movablerange of the scrub bar 53 are set to the same length by multiplyinglength in the horizontal direction of the area of the Fy-file displaysection 311, in which the variation graph is displayed, by apredetermined number, a position of the position indicator 312 displayedin the Fy-file display section 311 and a position of the scrub dial 54coincide with each other.

Therefore, it may be considered that the front frame to the end frame ofthe moving image to be subjected to scrub correspond to the respectivepositions in the horizontal direction of the area of the Fy-file displaysection 311 in which the variation graph is displayed. The positionindicator 312 is displayed in a position corresponding to the frame (theframe of interest) displayed in the viewer 51 among the positions in thehorizontal direction of the area of the Fy-file display section 311 inwhich the variation graph is displayed.

The range designation indicators 351S and 351E are GUIs of segmentsextending in the vertical direction and parallel to the signal indicator312. The range designation indicators 351S and 351E are displayed in thegraph display section 321 of the Fy-file display section 311. Moreover,the user can move the range designation indicators 351S and 351E in thehorizontal direction in the area in which the variation graph of (thegraph display section 321 of) the Fy-file display section 311 byoperating the keyboard 2 or the mouse 3.

However, movement of the range designation indicator 351S further to theright side than the range designation indicator 351E, in other words,movement of the range designation indicator 351E further to the leftside than the range designation indicator 351S is limited. Therefore, itis possible to move the range designation indicator 351S in a range fromthe left end of the area of the Fy-file display section 311 in which thevariation graph is displayed to a position of the range designationindicator 351E. It is possible to move the range designation indicator351E in a range from a position of the range designation indicator 351Sto the right end of the area of the Fy-file display section 311 in whichthe variation graph is displayed.

According to the range designation indicators 351S and 351E, among theframes forming the moving image to be subjected to scrub, frames in arange (an allocation range) from a frame corresponding to a position ofthe range designation indicator 351S to a frame corresponding to aposition of the range designation indicator 351E are allocated to thefront designated section to the end designated section in the movablerange of the scrub bar 53.

Therefore, when the scrub dial 54 is located at the left end of themovable range, a frame corresponding to a position of the rangedesignation indicator 351S is set as a frame of interest displayed onthe viewer 51. When the scrub dial 54 is located at the right end of themovable range, a frame corresponding to a position of the rangedesignation indicator 351E is set as a frame of interest displayed inthe viewer 51.

In a state in which the range designation indicator 351S is located atthe left end of the area of the Fy-file display section 311 in which thevariation graph is displayed and the range designation indicator 351E islocated at the right end of the area of the Fy-file display section 311in which the variation graph is displayed, the front designated sectionthe end designated section in the movable range of the scrub bar 53 areallocated to the front frame to the end frame of the moving image to besubjected to scrub. Therefore, this state is the default allocationstate.

In the default allocation state, when the scrub dial 54 is located atthe end of the movable range, the front frame of the moving image to besubjected to scrub is set as a frame of interest displayed in the viewer51. When the scrub dial 54 is located at the right end of the movablerange, the end frame of the moving image to be subjected to scrub is setas a frame of interest displayed in the viewer 51.

In the default allocation state, as described above, the frontdesignated section to the end designated section in the variable rangeof the scrub bar 53 are allocated to the front frame to the end frame ofthe moving image to be subjected to scrub.

Therefore, for example, when the number of frames of the moving image tobe subjected to scrub is large, a section length of designated sectionsis short. Thus, even when the scrub dial 54 is moved a little, a frameset as a frame of interest displayed in the viewer 51 substantiallychanges before and after the movement of the scrub dial 54 (a frameallocated to a designated section in which the scrub dial 54 after themovement is located is apart a larger number of frames from a frameallocated to a designated section, in which the scrub dial 54 before themovement is located). As a result, it may be difficult to find a frameof a desired image.

Thus, the range designation indicator 351S or 351E is moved to allocateframes in a range (an allocation range) from a frame corresponding to aposition of the range designation indicator 351S to a framecorresponding to a position of the range designation indicator 351Eamong the frames forming the moving image to be subjected to scrub tothe front designated section to the end designated section in themovable range of the scrub bar 53. This makes it possible to easily finda frame of a desired image.

By moving the range designation indicator 351S or 351E, it is possibleto set, as an allocation range (a range of frames allocated to the frontdesignated section to the end designated section in the movable range ofthe scrub bar 53), rather than the entire range, a part of a range froma frame corresponding to a position of the range designation indicator351S to a frame corresponding to a position of the range designationindicator 351E among the frames forming the moving image to be subjectedto scrub, i.e., a front frame to an end frame forming the moving imageto be subjected to scrub.

By setting a part of the range of the moving image to be subjected toscrub as an allocation range, compared with the case in which the entirerange is set as an allocation range, the number of frames allocated fromthe front designated section to the end designated section in themovable range of the scrub bar 53 is reduced. As a result, a sectionlength of the designated sections is increased.

Therefore, it is possible to prevent a frame set as a frame of interestdisplayed in the viewer 51 from substantially changing before and afterthe movement of the scrub dial 54 because of shortness of a sectionlength of the designated sections. As a result, it is possible to easilyfind a frame of a desired image.

When there is no motion in an image near a frame allocated to adesignated section in which the scrub dial 54 is located, even if aframe set as a frame of interest displayed in the viewer 51substantially changes before and after the movement of the scrub dial54, an image displayed in the viewer 51 does not change. Thus, even ifthe frame set as the frame of interest displayed in the viewer 51substantially changes before and after the movement of the scrub dial54, this does not hinder the user from finding a desired image.

On the other hand, when there is intense motion of an image near a frameallocated to a designated section in which the scrub dial 54 is located,when a frame set as a frame of interest displayed in the viewer 51substantially changes before and after the movement of the scrub dial54, an image displayed in the viewer 51 substantially changes. Thus, ifthe frame set as the frame of interest in the viewer 51 substantiallychanges before and after the movement of the scrub dial 54, this hindersthe user from finding a desired image.

Therefore, it is effective to set a section of frames having intensemotion of images of the moving image to be subjected to scrub as anallocation range (it is possible to easily find a frame of a desiredimage by setting a section of frames having intense motion of images ofthe moving image to be subjected to scrub as an allocation range).

In the Fy-file display section 311, if motion information is stored inan Fy file as a variation, a graph of the motion information (a motioninformation graph or a variation graph) is displayed and typeinformation representing a display type determined on the basis of themotion information is displayed.

The user can easily recognize the section having intense motion ofimages looking at the motion information graph and the type informationdisplayed in the Fy-file display section 311. Moreover, the user canoperate the range designation indicators 351S and 351E to set thesection as an allocation range.

Since the range designation indicators 351S and 351E indicate theallocation range that is the range of the frames allocated to the frontdesignated section to the end designated section in the movable range ofthe scrub bar 53, the range designation indicators 351S and 351E mayalso be referred to as allocation range information. Further, since therange designation indicators 351S and 351E are operated in designatingan allocation range, the range designation indicators 351S and 351E maybe referred to as range designation operating means.

The range designation indicator 351S of the range designation indicators351S and 351E is hereinafter also referred to as a start point indicator351S as appropriate. The range designation indicator 351E is hereinafteralso referred to as an end point indicator 351E as appropriate.

Allocation range setting processing for setting a range (an allocationrange) of frames allocated to (designated sections) of the movable rangeof the scrub bar 53 according to the range start point indicator 351Sand the end point indicator 351E as explained with reference to FIG. 61will be explained with reference to a flowchart in FIG. 62.

In step S371, the GUI control unit 211 (FIG. 43) controls the displaycontrol unit 218 to cause the Fy-file display section 311 to display thestart point indicator 351S and the end point indicator 351E. Theprocessing proceeds to step S372.

In step S371 performed immediately after the start of the allocationrange setting processing, the start point indicator 351S and the endpoint indicator 351E are displayed in positions at the left end and theright end of the area of the Fy-file display section 311 in which thevariation graph is displayed, respectively. Alternatively, the startpoint indicator 351S and the end point indicator 351E are displayed inpositions identical with positions of the start point indicator 351S andthe end point indicator 351E at the time when execution of the editingprogram was finished last time.

In step S372, the GUI control unit 211 (FIG. 43) sets a range from aframe corresponding to a position of the start point indicator 351S to aframe corresponding to a position of the end point indicator 351E amongthe frames forming the moving image to be subjected to scrub. The GUIcontrol unit 211 allocates the frames in the allocation range to themovable range of the scrub bar 53.

If weighting to the length of the designated sections is not taken intoaccount for simplification of the explanation, the GUI control unit 211divides the movable range of the scrub bar 53 into designated sectionshaving a quotient obtained by dividing the length of the movable rangeof the scrub bar 53 by the number of frames in the allocation range. TheGUI control unit 211 allocates the frames in the allocation range to thefront designated section to the end designated section in the movablerange.

The processing proceeds from step S372 to step S373. The GUI controlunit 211 judges whether movement of the start point indicator 351S orthe end point indicator 351E (FIG. 61) is started, i.e., whether theuser has operated the keyboard 2 or the mouse 3 to move the start pointindicator 351S or the end point indicator 351E.

When it is judged in step S373 that the movement of both the start pointindicator 351S and the end point indicator 351E is not started, theprocessing returns to step S373.

When it is judged in step S373 that the movement of the start pointindicator 351S or the end point indicator 351E is started, this meansthat the user has operated the keyboard 2 or the mouse 3 to move thestart point indicator 351S or the end point indicator 351E and anoperation signal corresponding to the operation is supplied from thekeyboard 2 or the mouse 3 to the GUI control unit 211. In this case, theGUI control unit 211 calculates, according to the operation signal fromthe keyboard 2 or the mouse 3, a position to which the start pointindicator 351S or the end point indicator 351E is moved. The GUI controlunit 211 supplies information representing the position to the displaycontrol unit 218. The processing proceeds to step S374.

In step S374, the display control unit 218 causes the start pointindicator 351S or the end point indicator 351E in a positioncorresponding to the information from the GUI control unit 211 insteadof the position where the indicator was displayed immediately before thestep. The processing proceeds to step S375.

In step S375, the GUI control unit 211 judges whether the movement ofthe start point indicator 351S or the end point indicator 351E isfinished, i.e., whether the user has finished the operation of thekeyboard 2 or the mouse 3 for moving the start point indicator 351S orthe end point indicator 351E.

When it is judged in step S375 that the operation for moving the startpoint indicator 351S or the end point indicator 351E is not finished,this means that the user continues to operate the keyboard 2 or themouse 3 to move the start point indicator 351S or the end pointindicator 351E and an operation signal corresponding to the operation issupplied from the keyboard 2 or the mouse 3 to the GUI control unit 211.In this case, the GUI control unit 211 calculates, according to theoperation signal from the keyboard 2 or the mouse 3, a position to whichthe start point indicator 351S or the end point indicator 351E is movedand supplies information representing the position to the displaycontrol unit 218. The processing returns to step S374. Thereafter, thesame processing is repeated.

According to the processing in steps S373 to S375, the start pointindicator 351S or the end point indicator 351E is moved in thehorizontal direction in accordance with the operation for moving thestart point indicator 351S or the end point indicator 351E.

On the other hand, when it is judged in step S375 that the operation formoving the start point indicator 351S or the end point indicator 351E isfinished, this means that the user has finished the operation of thekeyboard 2 or the mouse 3 for moving the start point indicator 351S orthe end point indicator 351E and an operation signal corresponding tothe operation for moving the start point indicator 351S or the end pointindicator 351E is not supplied from the keyboard 2 or the mouse 3 to theGUI control unit 211. In this case, the GUI control unit 211 recognizesa position (in the horizontal direction) of the start point indicator351S or the end point indicator 351E after the movement. The processingreturns to step S372. The GUI control unit 211 sets a range from a framecorresponding to a position of the start point indicator 351S to a framecorresponding to a position of the end point indicator 351E among theframes forming the moving image to be subjected to scrub as anallocation range. The GUI control unit 211 allocates frames in theallocation range to the movable range of the scrub bar 53. Thereafter,the same processing is repeated.

As described above, the user can easily change a range (an allocationrange) of frames to be subjected to scrub (frames allocated to themovable range of the scrub bar 53) among the frames forming the movingimage by operating the start point indicator 351S or the end pointindicator 351E serving as the GUI.

The allocation range setting processing in FIG. 62 ends when, forexample, the editing program ends.

In scrub, it is possible to change a size of an image of a framedisplayed in the viewer 51 according to, for example, a display type forthe frame.

FIG. 63 is a diagram showing an example of display of the viewer 51 atthe time when a size of an image of a frame displayed in the viewer 51is changed according to a display type for the frame.

The still image type V1, the normal type V2, or the high displayrate/low resolution type V3 explained with reference to FIG. 20 isdetermined as the display type for the frame on the basis of, forexample, motion information.

A first diagram from the left in FIG. 63 shows an example of display atthe time when an image of a frame of the still image type V1 isdisplayed in the viewer 51.

In the editing system in FIG. 43, for example, a default size is set inadvance for the image of the frame of the still image type V1. Theviewer 51 displays the image of the frame of the still image type V1with diagonal length or horizontal and vertical lengths adjusted to thedefault size set in advance.

A second diagram from the left in FIG. 63 shows an example of display atthe time when an image of a frame of the normal type V2 is displayed inthe viewer 51.

In the editing system in FIG. 43, for example, a size 1.5 times as largeas the default size is set for the image of the frame of the normal typeV2. The viewer 51 displays the image of the frame of the normal type V2with diagonal length or horizontal and vertical lengths adjusted to thesize 1.5 times as large as the default size.

A third diagram from the left (a first diagram from the right) in FIG.63 shows an example of display at the time when an image of a frame ofthe high display rate/low resolution type V3 is displayed in the viewer51.

In the editing system in FIG. 43, for example, a size twice as large asthe default size is set for the image of the frame of the high displayrate/low resolution type V3. The viewer 51 displays the image of theframe of the high display rate/low resolution type V3 with length ofdiagonal length or horizontal and vertical lengths adjusted to the sizetwice as large as the default size.

Since the image of the frame of the normal type V2 has a certain degreeof motion, by displaying the image of the frame of the normal type V2 inthe large size as described above, the user can check details of themotion of the image.

Since the image of the frame of the high display rate/low resolutiontype V3 has intense motion, by displaying the image of the frame of thehigh display rate/low resolution type V3 in the larger size, the usercan also check details of the motion of the image.

Moreover, as described above, by displaying an image in the sizedifferent for each of the display types, the user can easily recognize adisplay type for a frame of an image displayed in the viewer 51 and adegree of motion as well.

In FIG. 63, an image is displayed in the size different for each of thedisplay types determined on the basis of motion information. Besides,for example, it is also possible to display an image in a size differentfor each of display types determined on the basis of finenessinformation.

When an image is displayed in the size different for each of the displaytypes determined on the basis of fineness information, for example, bydisplaying an image of a display type determined on the basis offineness information with a larger degree of fineness in a larger size,the user can easily check details of the image.

In the processing of scrub explained with reference to FIG. 48, indisplaying an image in step S222, it is possible to display the image ina size corresponding to a display type.

The processing (display processing) in step S222 in FIG. 48 fordisplaying an image in a size corresponding to a display type will beexplained with reference to a flowchart in FIG. 64.

For example, the still image type V1, the normal type V2, or the highdisplay rate/low resolution type V3 explained with reference to FIG. 20is determined as the display type as described above. As explained withreference to FIG. 63, a default size is set for an image of a frame ofthe still image type V1. For example, a size 1.5 times as large as thedefault size is set for an image of a frame of the normal type V2.Moreover, for example, a size twice as large as the default size is setfor an image of a frame of the high display rate/low resolution type V3.

In step S391, the display control unit 218 (FIG. 43) judges which of thestill image type V1, the normal type V2, and the high display rate/lowresolution type V3 a display type for a frame (a frame of interest)allocated to a designated section, in which the scrub dial 54 islocated, supplied from the display-type acquiring unit 215 (FIG. 43) is.

When it is judged in step S391 that the display type for the frame ofinterest is the still image type V1, the processing proceeds to stepS392. The display control unit 218 causes the viewer 51 to display animage corresponding to the image data stored in the frame buffer 218A(FIG. 43) in the default size as shown in the first diagram from theleft in FIG. 63.

When it is judged in step S391 that the display type for the frame ofinterest is the normal type V2, the processing proceeds to step S393.The display control unit 218 causes the viewer 51 to display an imagecorresponding to the image data stored in the frame buffer 218A in thesize 1.5 times as large as the default size as shown in the seconddiagram from the left in FIG. 63.

When it is judged in step S391 that the display type for the frame ofinterest is the high display rate/low resolution type V3, the processingproceeds to step S394. The display control unit 218 causes the viewer 51to display an image corresponding to the image data stored in the framebuffer 218A in the size twice as large as the default size as shown inthe third diagram from the left in FIG. 63.

When an image is displayed in a size larger than the default size, asize of the viewer 51 is also increased. As a result, a part of the areaof the edition window 301 that is visible when the viewer 51 isdisplayed in the default size becomes invisible behind the viewer 51having the increased size. Thus, when the size of the viewer 51 isincreased, it is possible to display an image in the viewer 51, forexample, in a semitransparent state. In this case, it is possible toprevent a part of the area of the edition window 301 that is visiblewhen the viewer 51 is displayed in the default size from becominginvisible behind the viewer 51 having the increased size.

It is possible to change a size of (an image displayed in) the viewer 51according to a display size as described above only when the scrub dial54 is operated (e.g., dragged) and set the size to the default size whenthe operation of the scrub dial 54 is stopped.

Moreover, it is possible to change the size of the image according tonot only a display type for the frame of interest but also operation ofthe scrub dial 54 by the user.

In general, in searching for a desired image, the user operates thescrub dial 54 to move quickly when the user is not so interested in animage displayed in the viewer 51. Conversely, when the user isinterested in the image, the user operates the scrub dial 54 to moveslowly.

Thus, when the scrub dial 54 is moved quickly, it is possible to displayan image in the default size. When the scrub dial 54 is moved slowly, itis possible to display an image in a size larger than the default size.In this way, the user can easily find a desired image.

In FIG. 63, since a size of the entire viewer 51 is changed, a size ofthe scrub bar 53 is changed. However, when the size of the scrub bar 53is changed during operation thereof, it may be difficult to operate thescrub bar 53. Thus, it is desirable not to change the size of the scrubbar 53 (e.g., it is desirable to change a size of only the displaysection 52 (FIG. 3) of the viewer 51).

The invention explained above, scrub is performed. However, theinvention is also applicable when variable speed reproduction (n-timesspeed reproduction) other than scrub is performed.

In this embodiment, as means operated when a frame displayed in theviewer 51 is designated, the scrub bar 53 having the scrub dial 54 isadopted and a frame displayed in the viewer 51 is designated accordingto a position of the scrub dial 54. Besides, it is possible to designatea frame displayed in the viewer 51 by operating, for example, a (real)so-called jog dial, shuttle ring, or the like (not a GUI).

In this embodiment, a variation (motion information or finenessinformation) representing a degree of change in image data of a movingimage is calculated for each of frames, a display type is determined foreach of the frames on the basis of the variation, and the various kindsof processing for scrub are performed using the display type. Besides,for example, when sound data accompanies a moving image, it is possibleto determined a display type on the basis of a level (power) itself ofthe sound data or a value representing a degree of change in the sounddata. For example, in a frame in which the level of the sound datachanges from small to large (there is a climax of sound), it is highlylikely that an image of an interesting scene is shown. Thus, forexample, when motion information is large to some extent, it is possibleto determine a display type having a high display rate as a display typefor the frame in which the level of the sound data changes from small tolarge. For example, when fineness information is large to some extent,it is possible to determine a display type having a high resolution as adisplay type for the frame.

In this embodiment, assuming that the optical disk 7 is a professionaldisc in which main line data and proxy data are recorded, scrub isperformed using the main line data and the proxy data recorded in theoptical disk 7, i.e., image data having two kinds of resolutions.Besides, it is possible to perform scrub using, for example, image datahaving one kind of resolution or three or more kinds of resolutions.

In this embodiment, in order to cope with the problem of seek, thedisplay-type determining unit 93 in FIG. 22 (the display-typedetermining unit 233 in FIG. 44) judges whether frames having avariation equal to or larger than a threshold or smaller than thethreshold continue by the minimum limit number of frames N (judgment oncontinuity). However, if it is unnecessary to cope with the problem ofseek, it is possible to determine a display type according to whether avariation is equal to or larger than a threshold or smaller than thethreshold.

In this embodiment, a variation is calculated for each of frames and adisplay type is determined for each of the frames on the basis of thevariation. However, it is also possible to calculate a variation forevery plural frames. It is also possible to determine a display type forevery plural frames.

FIG. 65 is a graph of a variation for, for example, (a unit of) each offrames forming one GOP as plural frames. In FIG. 65, the abscissaindicates a frame and the ordinate indicates a variation.

In FIG. 65, a bold line indicates a variation for each of the framesforming the one GOP. As the variation for each of the frames forming theone GOP, it is possible to adopt, for example, an average of variationin all the frames forming the one GOP, a variation in I pictures amongthe frames forming the one GOP, or the like.

As shown in FIG. 65, when a variation for each of the frames forming theone GOP is calculated, it is possible to determine a display type for (aunit of) each of the frames forming the one GOP on the basis of thevariation in the same manner as the determination of a display type foreach of frames from a variation for each of the frames.

It is possible to determine, for example, a display type for each of theframes forming the one GOP as the plural frames on the basis of not onlya variation for each of the frames forming the one GOP but also avariation for each of frames.

FIG. 66 is a diagram of an Fy file in which both a variable for each offrames and a display type for each of the frames are stored.

For example, when one GOP is formed by fifteen frames, it is possible todetermine, for example, a display type used in the fifteen framesforming the one GOP at a highest ratio as a display type for each of thefifteen frames forming the one GOP.

In this case, for example, in one GOP formed by a first frame to afifteenth frame from the top shown in the Fy file in FIG. 66, thedisplay type V2 is used at a highest ratio among the display types V1,V2, and V3 for each of the frames. Thus, the display type V2 isdetermined as a display type for the one GOP.

For example, concerning the ninth frame from the top shown in the Fyfile in FIG. 66, a display type for each of the frames is V1 but adisplay type for one GOP is V2.

As described above, when a variation and a display type are determinedfor, for example, each of the frames forming one GOP as plural framesand stored in the Fy file, it is possible to reduce a capacity (a filesize) of the Fy file compared with that in the case in which a variationand a display type for each of frames are stored in the Fy file.Further, it is possible to reduce burdens of processing necessary foranalysis of the Fy file (file parse).

In the case in which image data is encoded in the MPEG system at thetime of creation of an Fy file, when a variation and a display type foreach of frames forming one GOP are determined, it is possible to includethe variation and the display type in a GOP header in a stream obtainedas a result of the encoding in the MPEG system.

In this embodiment, since image data is encoded in the MPEG system andrecorded in the optical disk 7, the decoder 216 of the editing system inFIG. 43 needs to decode the image data in the MPEG system. In the MPEGsystem, for example, with fifteen frames set as one GOP, each of theframes is encoded as a picture of one of picture types, namely, an I(Intra) picture, a P (Predictive) picture, and a B (BidirectionallyPredictive) picture. The P picture and the B picture among the I, P, andB pictures are encoded using the I or P picture encoded earlier as areference image (an image as a basis for generating a predicted image).Thus, it is possible to decode the P picture and the B picture onlyafter decoding the reference image.

For example, when one GOP is formed by fifteen frames and each of thefifteen frames is represented by I, P, or B indicating a picture typeand a number indicating a display order, it is possible to represent anarrangement of the fifteen frames of the one GOP as, for example, B1,B2, 13, B4, B5, P6, B7, B8, P9, B10, B11, P12, B13, B14, and P15.

In the GOP formed by the fifteen frames B1 to P15, for example, the Ppicture P6 sixth from the top is encoded using the I picture 13 thirdfrom the top as a reference image. The P picture P9 ninth from the topis encoded using the P picture P6 sixth from the top as a referenceimage. The P picture P12 twelfth from the top is encoded using the Ppicture P9 ninth from the top as a reference image. The P picture P15fifteenth from the top is encoded using the P picture P12 twelfth fromthe top as a reference image. The B picture B13 thirteenth from the topis encoded using the P picture P12 twelfth from the top and the Ppicture P15 fifteenth from the top as a reference image.

In this case, for example, when the B picture B13 thirteenth from thetop is set as a frame of interest displayed in the viewer 15, it ispossible to decode the B picture B13 thirteenth from the top only afterdecoding the I picture I3 third from the top, decoding the P picture P6sixth from the top with reference the I picture I3 third from the top,decoding the P picture P9 ninth from the top with reference to the Ppicture P6 sixth from the top, decoding the P picture P12 twelfth fromthe top with reference to the P picture P9 ninth from the top, anddecoding the P picture P15 fifteenth from the top with reference to theP picture P12 twelfth from the top. This is because it may be impossibleto refer to the P picture P12 twelfth from the top and the P picture P15fifteenth from the top before the respective pictures are decoded inthis way. Therefore, the decoding of the B picture B13 takes time.

Thus, the P pictures P6, P9, P12, and P15 referred to for the decodingof the other pictures are stored in a separate file called a P-to-I fileas I pictures I6, I9, I12, and I15, respectively. The decoder 216 canperform decoding with reference to the pictures stored in the P-to-Ifile when necessary. In this case, for example, it is possible to decodethe B picture B13 in a short time with reference to the I pictures I12and I15 stored in the P-to-I file.

In the series of processing described above, the processing performed bythe CPU 22 (FIG. 2) executing the programs can be performed by dedicatedhardware as long as available.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations, and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. A data processing apparatus that processes imagedata of a moving image with a processor, the data processing apparatuscomprising: acquiring means for acquiring variations representingdegrees of change in pictures of the image data, the variationsincluding fineness information representing a degree of spatial changein a picture of the pictures, motion information representing a degreeof temporal change between sequential pictures of the pictures, or boththe fineness information and the motion information, the finenessinformation indicating a degree of change in a pixel value among pixelsforming the picture, and the motion information indicating a degree ofchange of content between the sequential pictures in a time direction;display-type determining means for determining, on the basis of thevariations, display types respectively representing display methods fordisplaying the pictures, the display methods defining a display rate, adisplay resolution, or both a display rate and a display resolution fordisplaying the pictures; a file creating unit configured to create afile which associates the pictures with the display types determined bythe display-type determining means, and to store the file in a memory;display-type acquiring means for acquiring, from the file created by thefile creating unit, a display type for a picture designated bypicture-designation operating means operated in designating a picture tobe displayed; and display control means for displaying, on a display,the designated picture according to a display method corresponding tothe display type acquired by the display-type acquiring means for thedesignated picture, such that the moving image is displayed on thedisplay according to a display rate, a display resolution or both adisplay rate and a display resolution that change(s) as the moving imageis displayed in accordance with display types acquired from the file forsequentially displayed pictures constituting the displayed moving image.2. A data processing apparatus that processes image data of a movingimage with a processor, the data processing apparatus comprising: anacquiring unit configured to acquire variations representing degrees ofchange in pictures of the image data, the variations including finenessinformation representing a degree of spatial change in a picture of thepictures, motion information representing a degree of temporal changebetween sequential pictures of the pictures, or both the finenessinformation and the motion information, the fineness informationindicating a degree of change in a pixel value among pixels forming thepicture, and the motion information indicating a degree of change ofcontent between the sequential pictures in a time direction; adisplay-type determining unit configured to determine, on the basis ofthe variations, display types respectively representing display methodsfor displaying the pictures, the display methods defining a displayrate, a display resolution, or both a display rate and a displayresolution for displaying the pictures; a file creating unit configuredto create a file which associates the pictures with the display typesdetermined by the display-type determining unit, and to store the filein a memory; a display-type acquiring unit configured to acquire, fromthe file created by the file creating unit, a display type for a picturedesignated by a picture-designation operating unit operated indesignating a picture to be displayed; and a display control unitconfigured to display, on a display, the designated picture according toa display method corresponding to the display type acquired by thedisplay-type acquiring unit for the designated picture, and configuredto display the moving image on the display according to a display rate,a display resolution or both a display rate and a display resolutionthat change(s) as the moving image is displayed in accordance withdisplay types acquired from the file for sequentially displayed picturesconstituting the displayed moving image.
 3. A data processing apparatusaccording to claim 1, wherein: the display types represent displaymethods in which resolutions in displaying pictures or display rates indisplaying pictures are different from one another; the variationsinclude both the fineness information and the motion information; andthe display-type determining means determines, on the basis of thevariations including both the fineness information and the motioninformation, display types respectively representing display methods fordisplaying the pictures, the display methods defining both a displayrate and a display resolution for displaying the pictures.
 4. A dataprocessing apparatus according to claim 1, wherein: the display typesrepresent display methods in which resolutions in displaying pictures ordisplay rates in displaying pictures are different from one another; thevariations are fineness information representing a degree of spatialchange in a picture, and the display-type determining means determines,when the fineness information of a picture indicates that a degree ofspatial change is large, a high resolution/low display rate type havinga high resolution or a low display rate as a display type for thepicture.
 5. A data processing apparatus according to claim 1, wherein:the display types represent display methods in which resolutions indisplaying pictures or display rates in displaying pictures aredifferent from one another; the variations are motion informationrepresenting a degree of temporal change in a picture, and thedisplay-type determining means determines, when the motion informationof a picture indicates that a degree of temporal change is large, a lowresolution/high display rate having a low resolution or a high displayrate as a display type for the picture.
 6. A data processing apparatusaccording to claim 1, wherein: the display types represent displaymethods in which resolutions in displaying pictures or display rates indisplaying pictures are different from one another; the display typesinclude a still image type that represents display in a still image, thedisplay-type determining means determines the still image type, whichrepresents display in a still image, as a display type for a picturehaving a small variation, and the display control means causes, when apicture in a section in which pictures of the still image type amongpictures in time series continue is designated by thepicture-designation operating means, the display to display a picturedisplayed immediately before the picture again.
 7. A data processingapparatus according to claim 1, wherein the picture-designationoperating means is a bar in which a dial is movable, pictures areallocated in time series to subsections dividing a movable range inwhich the dial is movable, and the display control means causes thedisplay to display a picture allocated to a subsection in which the dialis located.
 8. A data processing apparatus according to claim 7, furthercomprising movement judging means for judging whether the dial is movedat a period corresponding to a display type for the picture allocated tothe subsection in which the dial is located, wherein the display controlmeans causes, when it is judged by the movement judging means that thedial is moved, the display to display a picture allocated to asubsection in which the dial after the movement is located.
 9. A dataprocessing apparatus according to claim 7, further comprising movementjudging means for judging whether the dial is moved at a periodcorresponding to a display type for the picture allocated to thesubsection in which the dial is located, wherein the display controlmeans causes, when it is judged by the movement judging means that thedial is not moved, the display to display a picture displayedimmediately before the picture again.
 10. A data processing apparatusaccording to claim 7, wherein the movable range is divided intosubsections having an identical section length.
 11. A data processingapparatus according to claim 7, wherein the movable range is dividedinto subsections having a larger section length as a picture having alarger variation is allocated thereto.
 12. A data processing apparatusaccording to claim 7, wherein a front picture to an end picture ofpictures in time series forming a moving image are allocated to a frontsubsection to an end subsection in the movable range.
 13. A dataprocessing apparatus according to claim 7, further comprisingrange-designation operating means operated in designating an allocationrange that is a range of pictures in time series allocated to a frontsubsection to an end subsection in the movable range, wherein a frontpicture to an end picture in the allocation range are allocated to thefront subsection to the end subsection in the movable range.
 14. A dataprocessing apparatus according to claim 7, wherein the display controlmeans causes the display to display type information representingdisplay types of the pictures in time series together with positioninformation representing a position of a picture allocated to asubsection in which the dial is located.
 15. A data processing apparatusaccording to claim 7, wherein the display-type determining meansdetermines a display type on the basis of a comparison result obtainedby comparing variation in the pictures and predetermined thresholds, andthe display control means causes the display to further display a graphrepresenting the variation in the pictures in time series together withthreshold information representing the thresholds.
 16. A data processingapparatus according to claim 15, wherein the threshold information ismoved according to operation by a user, and the display-type determiningunit determines a display type again on the basis of a comparison resultobtained by comparing the variation in the pictures and thresholdsrepresented by the threshold information after the movement.
 17. A dataprocessing apparatus according to claim 16, wherein the display controlmeans causes the display to further display type informationrepresenting the variation in the pictures in time series together withthe graph representing the variation in the pictures in time series andthe threshold information and, when a display type is determined againby the display-type determining means, causes the display to display thetype information again.
 18. A data processing apparatus according toclaim 15, wherein fineness information representing a degree of spatialchange in a picture and motion information representing a degree oftemporal change in a picture are stored in the file as variations in aunit of picture, and the display control means causes the display todisplay a graph representing the fineness information in time series, agraph representing the motion information in time series, or both thegraph representing the fineness information in time series and the graphrepresenting the motion information in time series.
 19. A dataprocessing apparatus according to claim 18, further comprising selectionoperating means operated in selecting the graph representing thefineness information in time series, the graph representing the motioninformation in time series, or both the graph representing the finenessinformation in time series and the graph representing the motioninformation in time series to be displayed.
 20. A data processingapparatus according to claim 15, wherein the display control meanscauses the display to further display, together with the graphrepresenting the variation in the pictures in time series and thethreshold information, allocation range information representing anallocation range that is a range of pictures in time series allocated toa front subsection to an end subsection in the movable range, theallocation range information is moved according to operation by a user,and a front picture to an end picture in the allocation rangerepresented by the allocation range information are allocated to thefront subsection to the end subsection in the movable range.
 21. A dataprocessing apparatus according to claim 1, wherein a size in displayingthe picture of the display type is set in advance, and the displaycontrol means causes the display to display a picture in a size set fora display type of the picture in advance.
 22. A data processing methodof processing image data of a moving image by a processor, the dataprocessing method comprising: determining, on the basis of variations inpictures representing degrees of change in the image data, display typesrespectively representing display methods for displaying the pictures,the variations including fineness information representing a degree ofspatial change in a picture of the pictures, motion informationrepresenting a degree of temporal change between sequential pictures ofthe pictures, or both the fineness information and the motioninformation, the fineness information indicating a degree of change in apixel value among pixels forming the picture, the motion informationindicating a degree of change of content between the sequential picturesin a time direction, and the display methods defining a display rate, adisplay resolution, or both a display rate and a display resolution fordisplaying the pictures; creating a file which associates the pictureswith the determined display types, and storing the file in a memory;acquiring a display type, from the file, for a picture designated bypicture-designation operating means operated in designating a picture tobe displayed; and displaying, on a display, the designated pictureaccording to a display method corresponding to the display typeacquired, in the acquiring, for the designated picture, includingdisplaying the moving image on the display according to a display rate,a display resolution or both a display rate and a display resolutionthat change(s) as the moving image is displayed in accordance withdisplay types acquired from the file for sequentially displayed picturesconstituting the displayed moving image.
 23. A data processing apparatusthat processes image data of a moving image with a processor, the dataprocessing apparatus comprising: acquiring means for acquiring displaytypes representing display methods for displaying pictures, the displaytypes being determined on the basis of variations representing degreesof change in pictures of the image data, the variations includingfineness information representing a degree of spatial change in apicture of the pictures, motion information representing a degree oftemporal change between sequential pictures of the pictures, or both thefineness information and the motion information, the finenessinformation indicating a degree of change in a pixel value among pixelsforming the picture, the motion information indicating a degree ofchange of content between the sequential pictures in a time direction,and the display methods defining a display rate, a display resolution,or both a display rate and a display resolution for displaying thepictures; display-type acquiring means for acquiring, from a fileassociating the pictures with respective determined display types, adisplay type for a picture designated by picture-designation operatingmeans operated in designating a picture to be displayed on the basis ofthe display types; and display control means for displaying, on adisplay, the designated picture according to a display methodcorresponding to the display type acquired by the display-type acquiringmeans for the designated picture, such that the moving image isdisplayed on the display according to a display rate, a displayresolution or both a display rate and a display resolution thatchange(s) as the moving image is displayed in accordance with displaytypes acquired from the file for sequentially displayed picturesconstituting the displayed moving image.
 24. A data processing apparatusthat processes image data of a moving image with a processor, the dataprocessing apparatus comprising: an acquiring unit configured to acquiredisplay types representing display methods for displaying pictures, thedisplay types being determined on the basis of variations representingdegrees of change in pictures of the image data, the variationsincluding fineness information representing a degree of spatial changein a picture of the pictures, motion information representing a degreeof temporal change between sequential pictures of the pictures, or boththe fineness information and the motion information, the finenessinformation indicating a degree of change in a pixel value among pixelsforming the picture, the motion information indicating a degree ofchange of content between the sequential pictures in a time direction,and the display methods defining a display rate, a display resolution,or both a display rate and a display resolution for displaying thepictures; a display-type acquiring unit configured to acquire, from afile associating the pictures with respective determined display types,a display type for a picture designated by a picture-designationoperating unit operated in designating a picture to be displayed on thebasis of the display types; and a display control unit configured todisplay, on a display, the designated picture according to a displaymethod corresponding to the display type acquired by the display-typeacquiring unit for the designated picture, and configured to display themoving image on the display according to a display rate, a displayresolution or both a display rate and a display resolution thatchange(s) as the moving image is displayed in accordance with displaytypes acquired from the file for sequentially displayed picturesconstituting the displayed moving image.
 25. A data processing method ofprocessing image data of a moving image by a processor, the dataprocessing method comprising: acquiring a display type, from a file, fora picture designated by picture-designation operating means operated indesignating a picture to be displayed on the basis of display types thatare determined on the basis of variations representing degrees of changein pictures of the image data, the display types representing displaymethods for displaying the pictures, the variations including finenessinformation representing a degree of spatial change in a picture of thepictures, motion information representing a degree of temporal changebetween sequential pictures, or both the fineness information and themotion information, the fineness information indicating a degree ofchange in a pixel value among pixels forming the picture, the motioninformation indicating a degree of change of content between thesequential pictures in a time direction, the display methods defining adisplay rate, a display resolution, or both a display rate and a displayresolution for displaying the pictures, and the file associating thepictures with respective determined display types; and displaying, on adisplay, the designated picture according to a display methodcorresponding to the display type acquired in the acquiring for thedesignated picture, including displaying the moving image on the displayaccording to a display rate, a display resolution or both a display rateand a display resolution that change(s) as the moving image is displayedin accordance with display types acquired from the file for sequentiallydisplayed pictures constituting the displayed moving image.
 26. A dataprocessing apparatus that processes image data of a moving image with aprocessor, the data processing apparatus comprising: acquiring means foracquiring variations representing degrees of change in pictures of theimage data, in a unit of one or more pictures, the variations includingfineness information representing a degree of spatial change in apicture of the pictures, motion information representing a degree oftemporal change between sequential pictures of the pictures, or both thefineness information and the motion information, the finenessinformation indicating a degree of change in a pixel value among pixelsforming the picture, and the motion information indicating a degree ofchange of content between the sequential pictures in a time direction;display-type determining means for determining, on the basis of thevariations, display types respectively representing display methods fordisplaying pictures, in a unit of plural pictures, the display methodsdefining a display rate, a display resolution, or both a display rateand a display resolution for displaying the pictures; a file creatingunit configured to create a file which associates the pictures with thedisplay types determined by the display-type determining means, and tostore the file in a memory; display-type acquiring means for acquiring,from the file created by the file creating unit, a display type for apicture designated by picture-designation operating means operated indesignating a picture to be displayed; and display control means fordisplaying, on a display, the designated picture according to a displaymethod corresponding to the display type acquired by the display-typeacquiring means for the designated picture, such that the moving imageis displayed on the display according to a display rate, a displayresolution or both a display rate and a display resolution thatchange(s) as the moving image is displayed in accordance with displaytypes acquired from the file for sequentially displayed picturesconstituting the displayed moving image.
 27. A data processing apparatusthat processes image data of a moving image with a processor, the dataprocessing apparatus comprising: an acquiring unit configured to acquirevariations representing degrees of change in pictures of the image datain a unit of one or more pictures, the variations including finenessinformation representing a degree of spatial change in a picture of thepictures, motion information representing a degree of temporal changebetween sequential pictures of the pictures, or both the finenessinformation and the motion information, the fineness informationindicating a degree of change in a pixel value among pixels forming thepicture, and the motion information indicating a degree of change ofcontent between the sequential pictures in a time direction; adisplay-type determining unit configured to determine, on the basis ofthe variations, display types respectively representing display methodsfor displaying the pictures, in a unit of plural pictures, the displaymethods defining a display rate, a display resolution, or both a displayrate and a display resolution for displaying the pictures; a filecreating unit configured to create a file which associates the pictureswith the display types determined by the display-type determining unit,and to store the file in a memory; a display-type acquiring unitconfigured to acquire, from the file created by the file creating unit,a display type for a picture designated by a picture-designationoperating unit operated in designating a picture to be displayed; and adisplay control unit configured display, on a display, the designatedpicture according to a display method corresponding to the display typeacquired by the display-type acquiring unit for the designated picture,and configured to display the moving image on the display according to adisplay rate, a display resolution or both a display rate and a displayresolution that change(s) as the moving image is displayed in accordancewith display types acquired from the file for sequentially displayedpictures constituting the displayed moving image.
 28. A data processingapparatus that processes image data of a moving image with a processor,the data processing apparatus comprising: acquiring means for acquiringdisplay types representing display methods for displaying pictures, in aunit of picture, the display types being determined on the basis ofvariations in a unit or more of pictures representing degrees of changein pictures of the image data, the variations including finenessinformation representing a degree of spatial change in a picture of thepictures, motion information representing a degree of temporal changebetween sequential pictures of the pictures, or both the finenessinformation and the motion information, the fineness informationindicating a degree of change in a pixel value among pixels forming thepicture, the motion information indicating a degree of change of contentbetween the sequential pictures in a time direction, and the displaymethods defining a display rate, a display resolution, or both a displayrate and a display resolution for displaying the pictures; display-typeacquiring means for acquiring, from a file associating the pictures withrespective determined display types, a display type for a picturedesignated by picture-designation operating means operated indesignating a picture to be displayed on the basis of the display types;and display control means for displaying, on a display, the designatedpicture according to a display method corresponding to the display typeacquired by the display-type acquiring means for the designated picture,such that the moving image is displayed on the display according to adisplay rate, a display resolution or both a display rate and a displayresolution that change(s) as the moving image is displayed in accordancewith display types acquired from the file for sequentially displayedpictures constituting the displayed moving image.
 29. A data processingapparatus that processes image data of a moving image with a processor,the data processing apparatus comprising: an acquiring unit configuredto acquire display types representing display methods for displayingpictures, in a unit of picture, the display types being determined onthe basis of variations in a unit or more of pictures representingdegrees of change in pictures the image data, the variations includingfineness information representing a degree of spatial change in apicture of the pictures, motion information representing a degree oftemporal change between sequential pictures of the pictures, or both thefineness information and the motion information, the finenessinformation indicating a degree of change in a pixel value among pixelsforming the picture, the motion information indicating a degree ofchange of content between the sequential pictures in a time direction,and the display methods defining a display rate, a display resolution,or both a display rate and a display resolution for displaying thepictures; a display-type acquiring unit configured to acquire, from afile associating the pictures with respective determined display types,a display type for a picture designated by picture-designation operatingmeans operated in designating a picture to be displayed on the basis ofthe display types; and a display control unit configured to display, ona display, the designated picture according to a display methodcorresponding to the display type acquired by the display-type acquiringunit for the designated picture, and configured to display the movingimage on the display according to a display rate, a display resolutionor both a display rate and a display resolution that change(s) as themoving image is displayed in accordance with display types acquired fromthe file for sequentially displayed pictures constituting the displayedmoving image.
 30. A data processing apparatus that processes image dataof a moving image with a processor, the data processing apparatuscomprising: acquiring means for acquiring the image data; variationcalculating means for calculating a variation representing a degree ofchange in pictures of the image data in a unit of picture, the variationincluding fineness information representing a degree of spatial changein a picture of the pictures, motion information representing a degreeof temporal change between sequential pictures of the pictures, or boththe fineness information and the motion information, the finenessinformation indicating a degree of change in a pixel value among pixelsforming the picture, and the motion information indicating a degree ofchange of content between the sequential pictures in a time direction;and file creating means for creating a file in which the variation in aunit of picture is stored as a metafile in which metadata of the imagedata is stored, the file indicating display types defining a displayrate, a display resolution or both a display rate and a displayresolution that change(s) when the moving image is displayed, and whichare configured to be acquired from the file for sequentially displayedpictures constituting the moving image.
 31. A data processing apparatusthat processes image data of a moving image with a processor, the dataprocessing apparatus comprising: an acquiring unit acquiring the imagedata; a variation calculating unit calculating a variation representinga degree of change in pictures of the image data in a unit of picture,the variation including fineness information representing a degree ofspatial change in a picture of the pictures, motion informationrepresenting a degree of temporal change between sequential pictures ofthe pictures, or both the fineness information and the motioninformation, the fineness information indicating a degree of change in apixel value among pixels forming the picture, and the motion informationindicating a degree of change of content between the sequential picturesin a time direction; and a file creating unit creating a file in whichthe variation in a unit of picture is stored as a metafile in whichmetadata of the image data is stored, the file indicating display typesdefining a display rate, a display resolution or both a display rate anda display resolution that change(s) when the moving image is displayed,and which are configured to be acquired from the file for sequentiallydisplayed pictures constituting the moving image.
 32. A data processingapparatus according to claim 30, wherein the acquiring means acquiresimage data by imaging a subject.
 33. A data processing apparatusaccording to claim 32, further comprising: encoding means for encodingthe image data acquired by the acquiring means; and recording means forrecording, in a recording medium, a file in which the image data encodedis stored and the file in which the variation is stored.
 34. A dataprocessing apparatus according to claim 30, wherein the acquiring meansacquires image data by reading out the image data from a recordingmedium in which the image data is recorded.
 35. A data processingapparatus according to claim 34, wherein the image data recorded in therecording medium is encoded, the data processing apparatus furtherincludes: decoding means for decoding the encoded data acquired by theacquiring means; and recording means for recording, in the recordingmedium, the file in which the variation is stored, and the variationcalculating means calculates, in a unit of picture, a variationrepresenting a degree of change in the image data decoded by thedecoding means.
 36. A data processing apparatus according to claim 30,further comprising display-type determining means for determining, onthe basis of a variation in a picture, a display type, which representsa display method in displaying the picture, in a unit of picture,wherein the file creating means creates a file in which the variationand the display type in a unit of picture are stored as a metafile inwhich metadata of the image data is stored.
 37. A data processingapparatus according to claim 30, further comprising recording means forrecording, in a recording medium, the file in which the variation isstored.
 38. A data processing method of processing image data of amoving image by a processor, the data processing method comprising:calculating a variation representing a degree of change in the imagedata in a unit of picture, the variation including fineness informationrepresenting a degree of spatial change in a picture of the pictures,motion information representing a degree of temporal change betweensequential pictures of the pictures, or both the fineness informationand the motion information, the fineness information indicating a degreeof change in a pixel value among pixels forming the picture, and themotion information indicating a degree of change of content between thesequential pictures in a time direction; and creating a file in whichthe variation in a unit of picture is stored as a metafile in whichmetadata of the image data is stored, the file indicating display typesdefining a display rate, a display resolution or both a display rate anda display resolution that change(s) when the moving image is displayed,and which are configured to be acquired from the file for sequentiallydisplayed pictures constituting the moving image.
 39. A data processingapparatus that processes image data of a moving image with a processor,the data processing apparatus comprising: acquiring means for acquiringthe image data; variation calculating means for calculating, in a unitof picture, a variation representing a degree of change in the imagedata, the variation including fineness information representing a degreeof spatial change in a picture of the pictures, motion informationrepresenting a degree of temporal change between sequential pictures ofthe pictures, or both the fineness information and the motioninformation, the fineness information indicating a degree of change in apixel value among pixels forming the picture, and the motion informationindicating a degree of change of content between the sequential picturesin a time direction; display-type determining means for determining, onthe basis of the variation in a picture, a display type representing adisplay method for displaying the picture, in a unit of picture, thedisplay method defining a display rate, a display resolution, or both adisplay rate and a display resolution for displaying the pictures; andfile creating means for creating a file in which the display type in aunit of picture is stored as a metafile in which metadata of the imagedata is stored, the file indicating display types defining a displayrate, a display resolution or both a display rate and a displayresolution that change(s) when the moving image is displayed, and whichare configured to be acquired from the file for sequentially displayedpictures constituting the moving image.
 40. A data processing apparatusthat processes image data of a moving image with a processor, the dataprocessing apparatus comprising: an acquiring unit acquiring the imagedata; a variation calculating unit calculating, in a unit of picture, avariation representing a degree of change in the image data, thevariation including fineness information representing a degree ofspatial change in a picture of the pictures, motion informationrepresenting a degree of temporal change between sequential pictures ofthe pictures, or both the fineness information and the motioninformation, the fineness information indicating a degree of change in apixel value among pixels forming the picture, and the motion informationindicating a degree of change of content between the sequential picturesin a time direction; a display-type determining unit determining, on thebasis of the variation in a picture, a display type representing adisplay method for displaying the picture, in a unit of picture, thedisplay method defining a display rate, a display resolution, or both adisplay rate and a display resolution for displaying the pictures; and afile creating unit creating a file in which the display type in a unitof picture is stored as a metafile in which metadata of the image datais stored, the file indicating display types defining a display rate, adisplay resolution or both a display rate and a display resolution thatchange(s) when the moving image is displayed, and which are configuredto be acquired from the file for sequentially displayed picturesconstituting the moving image.
 41. A data processing method ofprocessing image data of a moving image by a processor, the dataprocessing method comprising: calculating, in a unit of picture, avariable representing a degree of change in the image data, the variableidentifying fineness information representing a degree of spatial changein a picture of the pictures, motion information representing a degreeof temporal change between sequential pictures of the pictures, or boththe fineness information and the motion information, the finenessinformation indicating a degree of change in a pixel value among pixelsforming the picture, and the motion information indicating a degree ofchange of content between the sequential pictures in a time direction;determining, on the basis of the variable of a picture, a display typerepresenting a display method for displaying the picture, in a unit ofpicture, the display method defining a display rate, a displayresolution, or both a display rate and a display resolution fordisplaying the picture; and creating a file in which the display type ina unit of picture is stored as a metafile in which metadata of the imagedata is stored, the file indicating display types defining a displayrate, a display resolution or both a display rate and a displayresolution that change(s) when the moving image is displayed, and whichare configured to be acquired from the file for sequentially displayedpictures constituting the moving image.
 42. A data processing apparatusthat processes image data of a moving image with a processor, the dataprocessing apparatus comprising: acquiring means for acquiring the imagedata; variation calculating means for calculating, in a unit of picture,a variation representing a degree of change in the image data, thevariation including fineness information representing a degree ofspatial change in a picture of the pictures, motion informationrepresenting a degree of temporal change between sequential pictures ofthe pictures, or both the fineness information and the motioninformation, the fineness information indicating a degree of change in apixel value among pixels forming the picture, and the motion informationindicating a degree of change of content between the sequential picturesin a time direction; and file creating means for creating a file inwhich the variation is stored as a metafile in which metadata of theimage data is stored, the file indicating display types defining adisplay rate, a display resolution or both a display rate and a displayresolution that change(s) when the moving image is displayed, and whichare configured to be acquired from the file for sequentially displayedpictures constituting the moving image.
 43. A data processing apparatusthat processes image data of a moving image with a processor, the dataprocessing apparatus comprising: an acquiring unit acquiring the imagedata; a variation calculating unit calculating, in a unit of picture, avariation representing a degree of change in the image data, thevariation including fineness information representing a degree ofspatial change in a picture of the pictures, motion informationrepresenting a degree of temporal change between sequential pictures ofthe pictures, or both the fineness information and the motioninformation, the fineness information indicating a degree of change in apixel value among pixels forming the picture, and the motion informationindicating a degree of change of content between the sequential picturesin a time direction; and a file creating unit creating a file in whichthe variation is stored as a metafile in which metadata of the imagedata is stored, the file indicating display types defining a displayrate, a display resolution or both a display rate and a displayresolution that change(s) when the moving image is displayed, and whichare configured to be acquired from the file for sequentially displayedpictures constituting the moving image.
 44. A data processing apparatusthat processes image data of a moving image with a processor, the dataprocessing apparatus comprising: acquiring means for acquiring the imagedata; variation calculating means for calculating, in a unit of one ormore pictures, a variation representing a degree of a change in theimage data, the variation including fineness information representing adegree of spatial change in a picture of the pictures, motioninformation representing a degree of temporal change between sequentialpictures of the pictures, or both the fineness information and themotion information, the fineness information indicating a degree ofchange in a pixel value among pixels forming the picture, and the motioninformation indicating a degree of change of content between thesequential pictures in a time direction; display-type determining meansfor determining, on the basis of the variation, a display typerepresenting a display method in displaying a picture, in a unit ofplural pictures, the display method defining a display rate, a displayresolution, or both a display rate and a display resolution fordisplaying the picture; and file creating means for creating a file inwhich the display type is stored as a metafile in which metadata of theimage data is stored, the file indicating display types defining adisplay rate, a display resolution or both a display rate and a displayresolution that change(s) when the moving image is displayed, and whichare configured to be acquired from the file for sequentially displayedpictures constituting the moving image.
 45. A data processing apparatusthat processes image data of a moving image with a processor, the dataprocessing apparatus comprising: an acquiring unit acquiring the imagedata; a variation calculating unit calculating, in a unit of one or morepictures, a variation representing a degree of a change in the imagedata, the variation including fineness information representing a degreeof spatial change in a picture of the pictures, motion informationrepresenting a degree of temporal change between sequential pictures ofthe pictures, or both the fineness information and the motioninformation, the fineness information indicating a degree of change in apixel value among pixels forming the picture, and the motion informationindicating a degree of change of content between the sequential picturesin a time direction; a display-type determining unit determining, on thebasis of the variation, a display type representing a display method indisplaying a picture, in a unit of plural pictures, the display methoddefining a display rate, a display resolution, or both a display rateand a display resolution for displaying the picture; and a file creatingunit creating a file in which the display type is stored as a metafilein which metadata of the image data is stored, the file indicatingdisplay types defining a display rate, a display resolution or both adisplay rate and a display resolution that change(s) when the movingimage is displayed, and which are configured to be acquired from thefile for sequentially displayed pictures constituting the moving image.